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PREFACE 


A Note to Students, Teachers, and other Readers 


Thank you for reading this short preface. Allow me to share a few key points about 
the text so that you may better understand what you will find beyond this page. 

This text deals with matrix algebra, as opposed to linear algebra. Without argu- 
ing semantics, | view matrix algebra as a subset of linear algebra, focused primarily 
on basic concepts and solution techniques. There is little formal development of the- 
ory and abstract concepts are avoided. This is akin to the master carpenter teaching 
his apprentice how to use a hammer, saw and plane before teaching how to make a 
cabinet. 

This book is intended to be read. Each section starts with “AS YOU READ” questions 
that the reader should be able to answer after a careful reading of the section even if 
all the concepts of the section are not fully understood. | use these questions as a daily 
reading quiz for my students. The text is written in a conversational manner, hopefully 
resulting in a text that is easy (and even enjoyable) to read. 

Many examples are given to illustrate concepts. When a concept is first learned, 
| try to demonstrate all the necessary steps so mastery can be obtained. Later, when 
this concept is now a tool to study another idea, certain steps are glossed over to focus 
on the new material at hand. | would suggest that technology be employed in a similar 
fashion. 

This text is “open.” If it nearly suits your needs as an instructor, but falls short in 
any way, feel free to make changes?. | will readily share the source files (and help you 
understand them) and you can do with them as you wish. | would find such a process 
very rewarding on my own end, and I would enjoy seeing this text become better and 
even eventually grow into a separate linear algebra text. | do ask that the Creative 
Commons copyright be honored, in that any changes acknowledge this as a source 
and that it only be used non commercially. 

This is the third edition of the Fundamentals of Matrix Algebra text. | had not 
intended a third edition, but it proved necessary given the number of errors found in 
the second edition and the other opportunities found to improve the text. It varies 
from the first and second editions in mostly minor ways. | hope this edition is “stable;” 
| do not want a fourth edition anytime soon. 

Finally, | welcome any and all feedback. Please contact me with suggestions, cor- 
rections, etc. 

Sincerely, 

Gregory Hartman 


1This edition contains some minor modifications and substantial additions from the Rose-Hulman Insti- 
tute of Technology mathematics faculty 
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SYSTEMS OF LINEAR EQUATIONS 


You have probably encountered systems of linear equations before; you can proba- 
bly remember solving systems of equations where you had three equations, three 
unknowns, and you tried to find the value of the unknowns. In this chapter we will 
uncover some of the fundamental principles guiding the solution to such problems. 

Solving such systems was a bit time consuming, but not terribly difficult. So why 
bother? We bother because linear equations have many, many, many applications, 
from business to engineering to computer graphics to understanding more mathemat- 
ics. And not only are there many applications of systems of linear equations, on most 
occasions where these systems arise we are using far more than three variables. (En- 
gineering applications, for instance, often require thousands of variables.) So getting 
a good understanding of how to solve these systems effectively is important. 


But don’t worry; we'll start at the beginning. 


1.1 Introduction to Linear Equations 


AS YOU READ 


1. What is one of the annoying habits of mathematicians? 
2. What is the difference between constants and coefficients? 


3. Cana coefficient in a linear equation be 0? 


We’ll begin this section by examining a problem you probably already know how 
to solve. 


Example 1 Suppose a jar contains red, blue and green marbles. You are told 
that there are a total of 30 marbles in the jar; there are twice as many red marbles as 
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green ones; the number of blue marbles is the same as the sum of the red and green 
marbles. How many marbles of each color are there? 


SOLUTION We could attempt to solve this with some trial and error, and we’d 
probably get the correct answer without too much work. However, this won’t lend 
itself towards learning a good technique for solving larger problems, so let’s be more 
mathematical about it. 

Let’s let rrepresent the number of red marbles, and let b and g denote the number 
of blue and green marbles, respectively. We can use the given statements about the 
marbles in the jar to create some equations. 

Since we know there are 30 marbles in the jar, we know that 


r+b+g=30. (1.1) 


Also, we are told that there are twice as many red marbles as green ones, so we know 
that 
r= 2g. (1.2) 


Finally, we know that the number of blue marbles is the same as the sum of the red 
and green marbles, so we have 
b=r-g. (1.3) 


From this stage, there isn’t one “right” way of proceeding. Rather, there are many 
ways to use this information to find the solution. One way is to combine ideas from 
equations 1.2 and 1.3; in 1.3 replace r with 2g. This gives us 


b = 2g +g = 3g. (1.4) 


We can then combine equations 1.1, 1.2 and 1.4 by replacing r in 1.1 with 2g as we did 
before, and replacing b with 3g to get 


r+b+g= 30 
2g + 3g +g = 30 
6g = 30 

g=5 (1.5) 


We can now use equation 1.5 to find r and b; we know from 1.2 that r = 2g = 10 
and then since r + b + g = 30, we easily find that b = 15. 


Mathematicians often see solutions to given problems and then ask “What if. . .?” 
It’s an annoying habit that we would do well to develop — we should learn to think 
like a mathematician. What are the right kinds of “what if” questions to ask? Here’s 
another annoying habit of mathematicians: they often ask “wrong” questions. That 
is, they often ask questions and find that the answer isn’t particularly interesting. But 
asking enough questions often leads to some good “right” questions. So don’t be afraid 
of doing something “wrong;” we mathematicians do it all the time. 

So what is a good question to ask after seeing Example 1? Here are two possible 
questions: 
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1. Did we really have to call the red balls “r”? Could we call them “q”? 
2. What if we had 60 balls at the start instead of 30? 


Let’s look at the first question. Would the solution to our problem change if we 
called the red balls q? Of course not. At the end, we’d find that q = 10, and we would 
know that this meant that we had 10 red balls. 

Now let’s look at the second question. Suppose we had 60 balls, but the other 
relationships stayed the same. How would the situation and solution change? Let’s 
compare the “original” equations to the “new” equations. 


Original New 
r+b+g=30|]r+b+g=60 
r=2g r=2g 
b=r+g b=r+g 


By examining these equations, we see that nothing has changed except the first 
equation. It isn’t too much of a stretch of the imagination to see that we would solve 
this new problem exactly the same way that we solved the original one, except that 
we'd have twice as many of each type of ball. 

A conclusion from answering these two questions is this: it doesn’t matter what we 
call our variables, and while changing constants in the equations changes the solution, 
they don’t really change the method of how we solve these equations. 

In fact, it is a great discovery to realize that all we care about are the constants and 
the coefficients of the equations. By systematically handling these, we can solve any 
set of linear equations in a very nice way. Before we go on, we must first define what 
a linear equation is. 


Definition 1 Linear Equation 


A linear equation is an equation that can be written in the 
form 

4X1 + 2X2 +--+ + QnXn =C 
where the x; are variables (the unknowns), the a; are 
coefficients, and c is a constant. 


A system of linear equations is a set of linear equations that 
involve the same variables. 


A solution to a system of linear equations is a set of values 
for the variables x; such that each equation in the system is 
satisfied. 


So in Example 1, when we answered “how many marbles of each color are there?,” 
we were also answering “find a solution to a certain system of linear equations.” 
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The following are examples of linear equations: 


2x + 3y —7z = 29 
7 
at z% + x3 — X4 + 17x5 = V —10 
yı + 14y, +4 =y +13- yı 


3t 
VIr + Ts + EF cos(45°) 


Notice that the coefficients and constants can be fractions and irrational numbers 
(like 7, y/—10 and cos(45°)). The variables only come in the form of a;x;; that is, just 
one variable multiplied by a coefficient. (Note that 3t Z 2t, just a variable multiplied 
by a coefficient.) Also, it doesn’t really matter what side of the equation we put the 
variables and the constants, although most of the time we write them with the vari- 
ables on the left and the constants on the right. 

We would not regard the above collection of equations to constitute a system of 
equations, since each equation uses differently named variables. An example of a sys- 


tem of linear equations is 


Xı— X: +X3 4X, = 1 
2x, + 3x2 + X4 = 25 


X2 T X3 = 10 


It is important to notice that not all equations used all of the variables (it is more 
accurate to say that the coefficients can be 0, so the last equation could have been 
written as Ox, + X2 + x3 + 0x, = 10). Also, just because we have four unknowns does 
not mean we have to have four equations. We could have had fewer, even just one, 
and we could have had more. 

To get a better feel for what a linear equation is, we point out some examples of 
what are not linear equations. 


axy+z=1 
5x? + 2y” = 100 


1 
= + yy +24z =3 
x 


sin? x, + cos? X2 = 29 
2% + In X: = 13 


The first example is not a linear equation since the variables x and y are multi- 
plied together. The second is not a linear equation because the variables are raised 
to powers other than 1; that is also a problem in the third equation (remember that 
1/x =x~1 and „/x = x‘/2). Our variables cannot be the argument of function like sin, 
cos or In, nor can our variables be raised as an exponent. 


1.2 Using Matrices To Solve Systems of Linear Equations 


At this stage, we have yet to discuss how to efficiently find a solution to a system 
of linear equations. That is a goal for the upcoming sections. Right now we focus on 
identifying linear equations. It is also useful to “limber” up by solving a few systems of 
equations using any method we have at hand to refresh our memory about the basic 
process. 


Exercises 1.1 


In Exercises 1 — 10, state whether or not the 12. 2x — 3y = 3 
given equation is linear. 3x + by = 8 
1. x+y+z=10 x y z = 1 
2. xy+yz+xz=1 13. 2x + by =- z = —4 
4x 5y 2z = 0 
3. —3x +9 = 3y — 5z +x — 7 
4. y5y + 7x = —1 x +y- z = 1 
14. 2x + y = 2 
5. (x—1)(x+1) =0 y + 2 = 0 
6. yx +x = 25 
15. A farmer looks out his window at his 
7. Xx +y+t=1 chickens and pigs. He tells his daugh- 
8. 1 +9 = 3 cos(y) — 5z ter that he sees 62 heads and 190 legs. 
$ How many chickens and pigs does the 
9. cos(15)y + 7 = —1 farmer have? 
10. 2* + 2” = 16 


16. A person buys 20 trinkets at a yard sale. 


In Exercises 11 — 14, solve the system of linear The cost of each trinket is either $0.30 
equations. or $0.65. If this person spends $8.80, 
x + y = -1 how many of each type of trinket do 

I ag L 3y = 8 they buy? 


1.2 Using Matrices To Solve Systems of Linear Equations 


AS YOU READ 


1. What is remarkable about the definition of a matrix? 
2. Vertical lines of numbers in a matrix are called what? 
3. In a matrix A, the entry as3 refers to which entry? 


4. What is an augmented matrix? 


In Section 1.1 we solved a linear system using familiar techniques. Later, we com- 
mented that in the linear equations we formed, the most important information was 
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the coefficients and the constants; the names of the variables really didn’t matter. In 
Example 1 we had the following three equations: 


r+b+g=30 
r=2g 
b=r+g 


Let’s rewrite these equations so that all variables are on the left of the equal sign 
and all constants are on the right. Also, for a bit more consistency, let’s list the variables 
in alphabetical order in each equation. Therefore we can write the equations as 


b + g +r = 30 
—- 29 + r= 0. (1.6) 
-b + g + r= 0 


As we mentioned before, there isn’t just one “right” way of finding the solution to 
this system of equations. Here is another way to do it, a way that is a bit different from 
our method in Section 1.1. 

First, lets add the first and last equations together, and write the result as a new 
third equation. This gives us: 


b+ g + r = 30 
= 2g + r = 0: 
29 + 2r = 30 


A nice feature of this is that the only equation with a b in it is the first equation. 
Now let’s multiply the second equation by —}. This gives 


b+ g + r = 30 
g — r/2 = 0 
29 + 2r = 30 


Our next goal is to get rid of g in the third equation. To do this, let’s multiply the 
second equation by —2 and add that to the third equation, replacing the third equa- 
tion. Our new system of equations now becomes 


b + g+ r = 30 
g — rR = 0. 
3r = 30 


The next step is to multiply the third equation by 1, which leads us to 


b +g+ r = 30 
g — rR = 0. (1.7) 
r = 10 


At this stage we have the system in row-echelon form (to be precisely defined later), 
and we are ready to strip off the values of r, g, and b, in that order, by a process called 
back substitution. 
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Back substitution starts with the bottom equation in (1.7), from which we see that 
r = 10. Armed with this information the second equation in (1.7) becomes g— 10/2 = 
0 org = 5. Now that we know r and g we can use the first equation in (1.7) to find 
b +5 + 10 = 30 or b = 15. Clearly we have discovered the same result as when we 
solved this problem in Section 1.1. 


An alternative to back substitution is to continue making algebraic recombinations 
of the equations (1.7) to isolate the values of b, g, and r. Specifically, in (1.7), multiply 
the third equation by i and add the result to the second equation. Also, multiply the 
third equation by —1 and add it to the first. This leaves us with 


b + g = 20 
g = 5 
r = 10 


Finally, multiply the second equation by —1 and add it to the first to obtain 


g = 5A (1.8) 


The value of b, g, and r are now laid bare. Again, we arrive at the same result as when 
we solved this problem in Section 1.1 or via the back substitution above. 


Let us return to the idea that all that really matters are the coefficients and the 
constants to the right of the equal signs. There is nothing special about the letters b, 
g and r; we could have used x, y and z or x1, X2 and x3. And even then, since we wrote 
our equations so carefully, we really didn’t need to write the variable names at all as 
long as we put things “in the right place.” 


Let’s look again at our system of equations in (1.6) and write the coefficients and 
the right-side constants in a rectangular array. This time we won’t ignore the zeros, 
but rather write them out. 


b + g + r = 30 1 1 1 30 
—- 29 + r= 0 < 0 —2 1 0 
-b + g +r = 0 —1 1 1 0 


Notice how even the equal signs are gone; we don’t need them, for we know that the 
last column contains the right-side coefficients. 


We have just created a matrix. The definition of matrix is remarkable only in how 
unremarkable it seems. 7 
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Definition 2 Matrix 


A matrix is a rectangular array of numbers. 


The horizontal lines of numbers form rows and the vertical 
lines of numbers form columns. A matrix with m rows and n 
columns is said to be an m x n matrix (“an m by n matrix”). 
If m = n we say that the matrix is square. 


The entries of an m x n matrix are indexed as follows: 


Amı Am Am3 


That is, a32 means “the number in the third row and second 
column.” 


In the future, we’ll want to create matrices with just the coefficients of the vari- 
ables for a system of linear equations and leave out the right-side constants. There- 
fore, when we include these constants, we often refer to the resulting matrix as an 
augmented matrix. 


We can use augmented matrices to find solutions to linear equations by using es- 
sentially the same steps we used above. Each row of the matrix corresponds to an 
equation. In the example that follows we will use a shorthand to describe the matrix 
(and so equation) operations we perform; let R; represent “row i” in the matrix (equiv- 
alently, the ith equation). We can write “add c times row i to row j, and replace row 
j with that sum” as “cR; + R; — R3.” The expression “R; <> R;” means “interchange 
row / and row j.” The expression “cR; — R;” means “multiply row i by the scalar c.” In 
this last row operation we always assume c Æ 0. Each of these row operations is an 
algebraically legitimate operation that preserves the truth of the equations. 


We now proceed with the example computation. The original equations (1.6) are 
written below on the left, the corresponding matrix on the right. Below each is a com- 
ment indicating the operation we will perform to move to the next step. 


b g 
a 2g a 
—b ae an g a! 


1.2 


30 


jo) 


Replace equation 3 with the sum 
of equations 1 and 3 


b+ g + 
= 2 g pals 
2g + 


Multiply equation 2 by —1/2 


b + g + 


r 
r/2 
2r 


30 
0 
30 
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1 1 1 30 
Oo -2 1 0 
—1 1 1 0 


Replace row 3 with the sum of 
rows 1 and 3. 
(Ri + R3 > R3) 


1 1 1 30 

0O —2 1 0 

O 2 2 30 
Multiply row 2 by —1/2 

(—$R2 => R2) 

1 1 1 30 

O 1 -1/2 0 

0 2 2 30 


Replace row 3 with the sum of 
(—2) times row 2 plus row 3 


Replace equation 3 with the sum 
of (—2) times equation 2 plus 


equation 3 (—2R2 + R3 — R3) 

b +g+ r = 30 1 1 1 30 
g — r2 = 0 O 1 -1/2 0 

3r = 30 0 0 3 30 


Multiply row 3 by £ 


(ER; > Rs) 
b + g + r = 30 1 1 1 30 
g — r2 = 0 O 1 -1/2 0 
r = 10 0 0 1 10 


The final matrix on the right above contains the same solution information as we 
have on the left in the form of equations, which is again equation (1.7); we can now 
proceed with back substitution as before. Recall that the first column of our matrices 
held the coefficients of the b variable; the second and third columns held the coeffi- 
cients of the g and r variables, respectively. Therefore, the first row of the matrix can 
be interpreted as “b + g + r = 30,” or more concisely, “b = 15.” 


Alternatively, we may continue these row operations in the fashion that led us from 
(1.7) to (1.8). We have 
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b +g+ r = 30 11 1 30 
g — r2 = 0 O 1 —1/2 0 
r = 10 0 0 1 10 


Replace row 2 with the sum of $ 
times row 3 plus row 2 
(—2R3 + R2 —> R2); replace row 
1 with the sum of (—1) times 
row 3 plus row 1 
((—1)R3 + Ri > Ri. 


Replace equation 2 with the sum 
of $ times equation 3 plus 
equation 2, replace equation 1 
with the sum of —1 times 
equation 3 plus equation 1. 


b + g = 20 1 1 0 20 
g = 0 1 0 5 
r = 10 0 O 1 10 
Replace equation 1 with the sum Replace row 1 with the sum of 
of —1 times equation 2 plus —1 times row 2 plus row 1 
equation 1. (—R2 + Ri > R1). 
b = 15 1 0 0 15 
g = 5 0 1 0 5 
r = 10 0 O 1 10 


We again arrive at (1.8) and can readily read off the solution. 

There is an essential observation concerning the above process: each row oper- 
ation is invertible. Specifically, the operation R; + Rj; can be inverted or “undone” 
by doing it again, as R; + Rj. The operation cR; — R; is inverted by İR; — R; (since 
c # 0). The operation cR;+ Rj — Rj is inverted by —cR;+ Rj — Rj. As a result each suc- 
cessive system we obtain by applying these row operations can be transformed back 
to the previous system, and so the successive systems obtained are, algebraically, en- 
tirely equivalent. As a result any solution to one system is necessarily a solution to the 
others. 

Let’s practice this manipulation again. 


Example 2 Find a solution to the following system of linear equations by si- 
multaneously manipulating the equations and the corresponding augmented matri- 
ces. 
X1 + xX + XxX% = O 
2x, + 2x. + x3 = 0 
—1xı i X2 = 2x3 = 2 
SOLUTION We'll first convert this system of equations into a matrix, then we'll 


proceed by manipulating the system of equations (and hence the matrix) to find a 
solution. Again, there is not just one “right” way of proceeding. We will show how 
to obtain the solution via “back substitution” as above, and also how to continue the 
process so that back substitution is not required (in this example). 

The given system and its corresponding augmented matrix are seen below. 


10 
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Original system of equations 


X1 + xX + X% = O 1 1 1 0 
2x, + 2x. + X% = 0 2 2 1 0 
—1x, + Xx — 2x3 = 2 —1 1 —2 2 


Corresponding matrix 


We’ll proceed by trying to get the x; out of the second and third equation. 


Replace equation 2 with the sum 
of (—2) times equation 1 plus 
equation 2; 

Replace equation 3 with the sum 
of equation 1 and equation 3 


Replace row 2 with the sum of 
(—2) times row 1 plus row 2 
(—2R1 + R2 > R2); 
Replace row 3 with the sum of 
row 1 and row 3 
(Ri + R3—> R3) 


Xi + % + %*%3 += O 1 1 1 0 
—x = 0 0 0 -1 0 
2x2 = X3 = 2 0O 2 -1 2 


Notice that the second equation no longer contains x2. We’ll exchange the order of 
the equations so that we can follow the convention of solving for the second variable 


in the second equation. 


Interchange equations 2 and 3 


Interchange rows 2 and 3 


Ro © R3 
Xx + x + xk = 0 1 1 1 0 
2x2 = X3 = 2 (0) 2 —1 2 
-x = 0 0 0 -1 0 
i Multiply row 2 by 4 
Multiply equation 2 by 5 (ÍR > Rp) 
X1 + X2 + X% = 0 ay a 
x — dx, = 1 0 1 -á 1 
-x = 0 0 0 —1 0 
, ; Multiply row 3 by —1 
Multiply equation 3 by —1 (—1R3 > R3) 
X + xX. + x3 = O 11 1 0 
%2 — Ix, = 1 0 1 —į 1 
x3 = 0 00 1 =O 


The system above is now in row echelon form (for which we’ll give a precise defi- 
nition shortly). At this point we can back substitute to find the value of each variable. 
The third equation on the left above (equivalently, the last row of the matrix on the 
right) shows that x, = 0. Moving up to the second equation or row and using x3 = 0 
shows that x2 = 1, and then the first equation becomes x; + 1 + 0 = 0, so xı = —1 
and we are done. 

Alternatively, we may continue with row operations on the last system above and 


simplify the system even more. 


11 
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12 


Replace equation 1 with the sum Replace row 1 with the sum of 
of (—1) times equation 3 plus (—1) times row 3 plus row 1 
equation 1; (—R3 + Ri > R1); 
Replace equation 2 with the sum Replace row 2 with the sum of $ 
of 5 times equation 3 plus times row 3 plus row 2 
equation 2 ($R3 + R2 —> R2) 
Xi + X = 0 1 1 0 0 
X2 = 1 O 1 0 1 
x% = 0 0010 


Notice how the second equation shows that x2 = 1. All that remains to do is to 
solve for x4. 


Replace equation 1 with the sum Replace row 1 with the sum of 
of (—1) times equation 2 plus (—1) times row 2 plus row 1 
equation 1 (—R2 + Ri + R1) 
xy = —1 1 0 0 -1 
X2 = 1 0O 1 O 1 
x3 = 0 0 0 1 0 
Obviously the equations on the left tell us that x; = —1, x2 = 1 and x3 = 0, and 


notice how the matrix on the right tells us the same information. 


The procedure above in which we get the system into a form suitable for back sub- 
stitution is an example of Gaussian Elimination with Back Subsitution. The variation 
in which we perform additional row operations (for example, in moving from (1.7) to 
(1.8)) is called Gauss-Jordan Elimination. Both are perfectly valid and lead to the solu- 
tion of the system. Many students find Gauss-Jordan appealing, in that no back substi- 
tution is required, but in fact Gauss-Jordan requires about twice as a many arithmetic 
operations as Gaussian Elimination for solving larger systems. This isn’t an issue for 
the small systems we’re looking at, and Gauss-Jordan can make certain intuitive and 
theoretical conclusions clearer, so we’ll continus to use both whereever appropriate. 
But be aware that modern software uses Gaussian Elimination with back substitution 
(with a few modifications) for solving general linear systems. 


Exercises 1.2 


In Exercises 1 — 4, convert the given system of X1 + 3X2 — 4x3 + 5x4 = 17 
linear equations into an augmented matrix. 3. —X1 + 4x3 + 8X4 = 1 
2X1 T 3x2 + 4x3 + 5X4 = 6 
3x + 4y + 5 = 7 
1L -x + y — 3z = 1 3x1 —2x2= 4 
2x — 2y + 3z = 5 4 2x,= 3 
i —Xı + 9x2 = 8 
2x + 5y — 6 2 5x1 —7X. = 13 
2 9x — 8z = 10 
—2x + 4y + z = -7 In Exercises 5 — 9, convert the given aug- 


1.2 Using Matrices To Solve Systems of Linear Equations 


mented matrix into a system of linear equa- 1 1 1 
tions. Use the variables x1, x2, etc. 17. B=|2 1 2 
z 1 2 3 
5 1 2 3 
“peed. 33. 39 3 5 7 
AT ME 
'| 0O 1 -2 
[1 1 -1 —1 2 1 0 1 
7. 13 5 7 19.B=]1 1 1 
- 1 2 3 
1000 2 
8 O 1 0 0 -1 1 1 1 
“10 O 1 0 5 20. B= |1 0 1 
0001 3 02 2 
9 1 0 0 7 2 In Exercises 21 — 26, rewrite the system of 
Os he BF 520. 5: equations in matrix form. Find the solution 


to the linear system by simultaneously ma- 
nipulating the equations and the matrix using 
Gaussian Elimination followed by back substi- 


In Exercises 10 — 15, perform the given row 
operations on A, where 


2 -1 7 tution. Repeat the solution procedure using 
A=1!10 4 -2]. Gauss-Jordan Elimination. 
a 2 21 x + y = 3 
“2x — 3y = 1 
10. —1Rı —> Ri 
11. R2 © R3 2x + 4y = 10 
22. ye a 
12. Ri Re Ri í 
13. 2R2 + R3 > R 
2 + R3 3 5 -2x + 3y = 2 
14. ER > Ro eX oy a 
15. —3Ri + R3 > R3 
A matrix A is given below. In Exercises 16- 24, 2X + 3y = 2 
20, a matrix B is given. Give the row opera- -2x + 6y = 1 
tion that transforms A into B. 
1 1 1 —5x1 + 2x3 = 14 
A=/]1 01 25. X2 = 1 
1 2 3 —3x1ı + x3 = 8 
1 1 1 — 5x. + 2x3 = -11 
16.B=/]2 0 2 26. xXx + 2x3 = 15 
1 2 3 — 3x2 + X3 = —8 
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1.3 Elementary Row Operations and Gaussian Elimina- 
tion 
1. Give two reasons why the Elementary Row Operations are called “Elementary.” 


2. T/F: Assuming a solution exists, all linear systems of equations can be solved 
using only elementary row operations. 


3. Give one reason why one might not be interested in putting a matrix into re- 
duced row echelon form. 


4. Identify the pivots in the following matrix: 


oOOORF 
ooro 
Orero 
OrROR 


5. Using the “forward” and “backward” steps of Gaussian elimination creates lots 
of making computations easier. 


In our examples thus far, we have essentially used just three types of manipulations 
in order to find solutions to our systems of equations. These three manipulations are: 


1. Add a scalar multiple of one equation to a second equation, and replace the 
second equation with that sum 


2. Multiply one equation by a nonzero scalar 


3. Swap the position of two equations in our list 


We saw earlier how we could write all the information of a system of equations in 
a matrix, so it makes sense that we can perform similar operations on matrices (as we 
have done before). Again, simply replace the word “equation” above with the word 
“row.” 

We didn’t justify our ability to manipulate our equations in the above three ways; 
it seems rather obvious that we should be able to do that. In that sense, these oper- 
ations are “elementary.” These operations are elementary in another sense; they are 
fundamental -they form the basis for much of what we will do in matrix algebra. Since 


these operations are so important, we list them again here in the context of matrices. 
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Key Idea 1 Elementary Row Operations 


1. Add ascalar multiple of one row to another row, and 
replace the latter row with that sum 


. Multiply one row by a nonzero scalar 


. Swap the position of two rows 


Given any system of linear equations, we can find any solution (if any exist) by 
using these three row operations. Elementary row operations give us a new linear 
system, but because these elementary operations are invertible, and the solution to 
the new system is also a solution to the old (and vice-versa). Thus we can use these 
Operations as much as we want and not change the solution. This brings to mind two 
good questions: 


1. Since we can use these operations as much as we want, how do we know when 
to stop? (Where are we supposed to “go” with these operations?) 


2. Is there an efficient way of using these operations? (How do we get “there” the 
fastest?) 


We’ll answer the first question first. Most of the time we will want to take our 
original matrix and, using the elementary row operations, put it into something called 
row echelon form (if we will perform back substitution) or reduced row echelon form 
(which allows us to avoid back substitution).? One of these forms will be our “destina- 
tion,” for either form allows us to readily identify whether or not solutions exist and if 
so, what these solutions are (there may be more than one). 


In the previous section, when we manipulated matrices to find solutions, we were 
unwittingly putting the matrix into row echelon form or reduced row echelon form. 
However, not all solutions come in such a simple manner as we’ve seen so far. Putting 
a matrix into either form helps us identify all types of solutions. We'll explore the topic 
of understanding what the row echelon or reduced row echelon form of a matrix tells 
us in the following sections; in this section we focus how to distill a matrix to one of 
these forms. 


1Some texts use the term reduced echelon form instead of reduced row echelon form. 
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Definition 3 Row Echelon Form 


A matrix is in row echelon form if its entries satisfy the fol- 
lowing conditions. 


1. The first nonzero entry in each rowis a 1 (called a pivot 
or leading 1). 


. Each pivot comes in a column to the right of the pivots 
in rows above it. 


. All rows of all Os come at the bottom of the matrix. 


. If a column contains a pivot, then all other entries be- 
low the pivot in that column are 0. 


A matrix that satisfies all these conditions and additionally 
has all entries above each pivot equal to zero is said to be in 
reduced row echelon form. 


Note that if a matrix is in reduced row echelon form it is automatically in row echelon 
form, but not vice-versa. 


Example 3 Which of the following matrices is in reduced row echelon form? 
Which is in row echelon form? Which is in neither form? 
1 0 1 0 1 
J | Jf: 1 | 


e] 
Ore 
e O 
—— 


O 
m. 
N 
w 
m. 
m. 
jo) 
m. 


Oo 
= 
j=) 
N 


oO 
m ees — 
OF 
O O 
oo 
Or 
= 
a So SS 
OF 
O N 
w o 
O O 


(=) 
(=) 
(=) 
(=) 

oero 

oup 
= 


SOLUTION The matrices in a), b), c), d) and g) are all in reduced row echelon 
form, and so also in row echelon form. The matrix h) is in row echelon form, but not 
reduced row echelon form. Check to see that each satisfies the necessary conditions. 
If your instincts were wrong on some of these, correct your thinking accordingly. 
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The matrix in e) is not in reduced row echelon form since the row of all zeros is 
not at the bottom. The matrix in f) is not in reduced row echelon form since the first 
nonzero entries in rows 2 and 3 are not 1. Finally, the matrix in h) is not in reduced row 
echelon form since the first entry in column 2 is not zero; the second 1 in column 2 is 
a pivot, hence all other entries in that column should be 0. However, since all entries 
in the column below this pivot are 0, the matrix is in row echelon form. 

We end this example with a preview of what we’ll learn in the future. Consider 
the matrix in b). If this matrix came from the augmented matrix of a system of linear 
equations, then we can readily recognize that the solution of the system is x; = 1 and 
X = 2. Similar, if the matrix in h) came the augmented matrix of a linear system we 
could back substitute to see that the solution is x3 = 3,x. = 2, and x; = —1. Again, 
in previous examples, when we found the solution to a linear system, we were un- 
wittingly putting our matrices into reduced row echelon form, or simply row echelon 
form. 


We began this section discussing how we can manipulate the entries in a matrix 
with elementary row operations. This led to two questions, “Where do we go?” and 
“How do we get there quickly?” We’ve just answered the first question: most of the 
time we are “going to row echelon form or reduced row echelon form. We now address 
the second question. 

There is no one “right” way of using these operations to transform a matrix into 
row echelon or reduced row echelon form. However, there is a general technique that 
works very well in that it is very efficient (so we don’t waste time on unnecessary steps). 
This technique is called Gaussian elimination or Gauss-Jordan elimination, depending 
on where we stop. It is named in honor of the great mathematician Karl Friedrich 
Gauss. 

While this technique isn’t very difficult to use, it is one of those things that is eas- 
ier understood by watching it being used than explained as a series of steps. With this 
in mind, we will go through one more example highlighting important steps and then 
we'll explain the procedure in detail. 


Example 4 Put the augmented matrix of the following system of linear equa- 
tions into row echelon form, and then into reduced row echelon form. 


—3x, — 3x. + 9x3 = 12 
2X, + 2x. — 4x3 = -2 
—2x> = 4x3 = —8 
SOLUTION We start by converting the linear system into an augmented ma- 


trix. 
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Our next step is to change the entry in the box to a 1. To do this, let’s multiply row 
1 by —2. 
3 


1 1 -3 -4 
—4Ri > Ri 2 2 —4 —2 
0 —2 -4 -8 


We have now created a pivot; that is, the first entry in the first row is a 1. Our next 
step is to put zeros under this pivot. To do this, we’ll use the elementary row operation 
given below. 


1 1 -3 —4 


—2Rı + R2 > R2 (0) (0) 2 6 
0 —2 -4 -8 


Once this is accomplished, we shift our focus from the pivot down one row, and to 
the right one column, to the position that is boxed. We again want to put a 1 in this 
position. We can use any elementary row operations, but we need to restrict ourselves 
to using only the second row and any rows below it. Probably the simplest thing we 
can do is interchange rows 2 and 3, and then scale the new second row so that there 
is a 1 in the desired position. 


pa 
m.e 


-3 —4 
Ry +> Rs o |-2]| -4 -8 


1 1-3 —4 
— $R — R2 O 1 2 4 
0 0 |2 6 


We have now created another pivot, this time in the second row. Our next desire 
is to put zeros underneath it, but this has already been accomplished by our previous 
steps. Therefore we again shift our attention to the right one column and down one 
row, to the next position put in the box. We want that to be a 1. A simple scaling will 
accomplish this. 


1 1 -3 —4 
5R3 — R3 0O 1 2 4 
0 0 1 3 


This ends what we will refer to as the forward steps that constitute Gaussian elim- 
ination and leave the matrix in row echelon form. At this stage we can back substitute 
to find solution x3 = 3, x2 = 4 — (2)(3) = —2, and x, = —4 + (3)(3) — (—2) = 7. 

If we wish to proceed with Gauss-Jordan elimination and get the system to reduced 
row echelon form, our next task is to use the elementary row operations and go back 
and put zeros above our pivots. This is referred to as the backward steps. These steps 
are given below. 


1.3 Elementary Row Operations and Gaussian Elimination 


3R3 + Rı > R Lae ? 
3 ul 1 

010 -2 
—2R R R 

3 +R2 > R2 001 3 

100 7 

—R + Ri > Ri O 1 0 -2 

0 0 1 3 

It is now easy to read off the solution as x; = 7, x2 = —2 and x3 = 3. 


We now formally explain the procedure used to find the solution above. As you 
read through the procedure, follow along with the example above so that the expla- 
nation makes more sense. 


Forward Steps 


1. Working from left to right, consider the first column that isn’t all zeros that hasn’t 
already been worked on. Then working from top to bottom, consider the first 
row that hasn’t been worked on. 


2. If the entry in the row and column that we are considering is zero, interchange 
rows with a row below the current row so that entry is nonzero. If all entries 
below are zero, we are done with this column; start again at step 1. 


3. Multiply the current row by a scalar to make its first entry a 1 (a pivot). 
4. Repeatedly use Elementary Row Operation 1 to put zeros underneath the pivot. 


5. Go back to step 1 and work on the new rows and columns until either all rows 
or columns have been worked on. 


If the above steps have been followed properly, then the following should be true 
about the current state of the matrix: 


1. The first nonzero entry in each row is a 1 (a pivot). 
2. Each pivot is in a column to the right of the pivots above it. 
3. All rows of all zeros come at the bottom of the matrix. 


Note that this means we have just put a matrix into row echelon form. We may 
back substitute here to find the solution. 

Alternatively, we can finish the conversion into reduced row echelon form; these 
additional steps constitute Gauss-Jordan elimination. These next steps are referred to 
as the backward steps. These are much easier to state. 


Backward Steps 


1. Starting from the right and working left, use Elementary Row Operation 1 re- 
peatedly to put zeros above each pivot. 
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The basic method of Gaussian elimination is this: create pivots and then use ele- 
mentary row operations to put zeros below the pivots. Gauss-Jordan elimination takes 
it further, by installing zeros above the pivots as well. We can do this in any order we 
please, but by following the “Forward Steps” and “Backward Steps,” we make use of 
the presence of zeros to make the overall computations easier. This method is very 
efficient, so it gets its own name (which we’ve already been using). 


Definition 4 Gaussian Elimination 


Gaussian elimination is the technique for finding the row 
echelon form of a matrix using the above procedure. It can 
be abbreviated to: 


. Create a pivot. 


. Use this pivot to put zeros underneath it. 


. Repeat the above steps until all possible rows have 
pivots. 


If we add a 4th step “Put zeros above these pivots,” we have Gauss-Jordan elimi- 
nation. 


Let’s practice some more. 


Example 5 Use Gaussian elimination to put the matrix A into row echelon and 
then reduced row echelon form, where 


SOLUTION We start by wanting to make the entry in the first column and first 
row a 1 (a pivot). To do this we'll scale the first row by a factor of —}. 


1 2 1 5 O 
— $R —> Ri 2 4 1 9 —2 
3 6 1 13 —4 


—2Rı + Rə > R2 
—3Rı + R3 —> R3 


j=) 
j=) 
l 
hb 
l 
e 
l 
N 


1.3 Elementary Row Operations and Gaussian Elimination 


Our attention now shifts to the right one column and down one row to the position 
indicated by the box. We want to put a 1 in that position. Our only options are to either 
scale the current row or to interchange rows with a row below it. However, in this case 
neither of these options will accomplish our goal. Therefore, we shift our attention to 
the right one more column. 

We want to put a 1 where there is a—1. A simple scaling will accomplish this; once 
done, we will put a O underneath this leading one. 


12 1 5 0 
—R2 > R2 0 o0 1 1 2 
0 0 -2 -2 —4 
121 5 0 
2R2 + R3 —> R3 O O 1 1 2 
0 0 0 O| O 


Our attention now shifts over one more column and down one row to the position 
indicated by the box; we wish to make this a 1. Of course, there is no way to do this, 
so we are done with the forward steps. The matrix is now in echelon form. You should 
check that the appropriate conditions of Definition 3 are met. In this example we will 
not back substitute. 

If we proceed with Gauss-Jordan elimination then our next goal is to put a 0 above 
each of the pivots (in this case there is only one pivot to deal with). 


1 2 
—R2 + Ri — Ri 0 o0 
0 0 


This final matrix is in reduced row echelon form. The issue of what solutions this 
might encode for the original system will be deferred for now. 


Example 6 Put the matrix 
1 2 1 3 
2 1 1 1 
3 3 2 1 
into reduced row echelon form. 
SOLUTION Here we will show all steps without explaining each one. 
IRER $ 1 2 1 3 
Za H Rs z R; oTa =L o5 
E 0 -3 -1 -8 
1 2 1 3 
0 -3 -1 -8 
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1 2 1 3 
3R2 + R3 + R3 0 1 1/3 5/3 
00 0 -3 
1 2 1 3 
—3R3 > R3 0 1 1/3 5/3 
00 0 1 
1 2 1 0 
—3R3+R1 > Rı 
0 1 1/3 0 
mee $ 0 : 1 
1 0 1/3 0 
—2R2 + Rı > Ry O 1 1/3 0 
00 0 1 


The last matrix in the above example is in reduced row echelon form. The matrix 
two steps prior is in row echelon form. If one thinks of the original matrix as represent- 
ing the augmented matrix of a system of linear equations, this final result is interesting. 
What does it mean to have a leading one in the last column? We'll figure this out in 
the next section. 


Let’s do one final example. 


Example 7 Put the matrix A into row echelon form and reduced row eche- 
lon form, where 


SOLUTION We'll again show the steps without explanation. 


1 1/2 172 2 


ER > Ri 1 2 2 12 

2 —1 9 

Ri +R >R ua ma a 

=ni 2 2 

as =e 5/2 10 
0 5 

1 1/2 -1/2 2 

Oo 1 0 5 


jä 


1/2 -1/2 2 
0 0 5/3 35/3 
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1 1/2 —1/2 2 
o o0 1 7 
We know have the matrix in row echelon form. If the original matrix was the aug- 
mented matrix for a linear system then we’ve now show this system is equivalent to 


Xı +1/2x,—1/2x3 = 2 
X2 — 5/3x3 = —20/3 i 
X3 = 7 


We can back substitute to find that x3 = 7, so the second equation turns into 
x — (5/3)(7) = —20/3, 


telling us that x. = 5. Finally, knowing values for x2 and x3 lets us substitute in the first 
equation and find 


xı + (1/2)(5) — (1/2)(7) = 2, 


sox, = 3. 


If we continue with Gauss-Jordan elimination we find 


SR3 + Ry > Ra 1 1/2 -1/2 2 
(knowing x3 = 7 allows us Oo 1 (0) 5 
to find x = 5) (0) (0) 1 7 


ÍR + Ri > Ri 
— $R + Ri > Ri 
(knowing x2 = 5 and x3 = 7 
allows us to find x; = 3) 


OOF 
Oro 
POO 
NUW 


Uniqueness of Row Echelon and Reduced Row Echelon Form: There is one last fact 
worth mentioning, but that we won’t stop to prove formally: A matrix typically has 
many different row echelon forms, and many different sequences of row operations 
lead to each. It doesn’t matter which one we use, for all encode the same solution 
set to the original linear equations. However, a given matrix has a unique reduced 
row echelon form. There many be many sequences of row operations that lead to this 
reduced row echelon form, but all result in the same final matrix. 


In all of our practice, we’ve only encountered systems of linear equations with ex- 
actly one solution. Is this always going to be the case? Could we ever have systems 
with more than one solution? If so, how many solutions could there be? Could we 
have systems without a solution? These are some of the questions we’ll address in 
the next section. 


Exercises 1.3 
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In Exercises 1 — 4, state whether or not the 
given matrices are in reduced row echelon 
form, row echelon form, or neither. In each 


case state why. 


(d) 


1. (a) a 
A 

2. (a) A 
(b) | 5 

1 

3. (a) | 0 
0 

1 

(b) | 0 

0 

1 

(c) | 0 

0 

1 

(d) | 0 

0 

2 

4. (a) | 0 
0 

0 

(b) | 0 

0 

0 

(c) 10 

0 

1 

0 

0 


In Exercises 5 — 22, use Gaussian Elimination 
to put the given matrix into both row echelon 


b | 


eO OO ore RPO 


oor OOO OOF ONO OBRO OOO OBRO Orere 


ere eO 


POO OrFO OOO ereere 


eO OOrRF OBRO NOO 


oO 


1 


1 


1 


1 


oO FO Ore ere 


OOO NNN 


and reduced row echelon form. 


1 2 
[4 


2 
s [2 
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1.4 Existence and Uniqueness of Solutions 


1.4 Existence and Uniqueness of Solutions 


AS YOU READ 


1. T/F: It is possible for a linear system to have exactly 5 solutions. 

2. T/F: A variable that corresponds to a pivot is “free.” 

3. How can one tell what kind of solution set a linear system of equations has? 
4 


. Give an example (different from those given in the text) of a 2 equation, 2 un- 
known linear system that is not consistent. 


5. T/F: A particular solution for a linear system with infinitely many solutions can 
be found by arbitrarily picking values for the free variables. 


So far, whenever we have solved a system of linear equations, we have always 
found exactly one solution. This is not always the case; we will find in this section that 
some systems do not have a solution, and others have more than one. 

We start with a very simple example. Consider the following linear system: 


x—y=0. 


There are obviously infinitely many solutions to this system; as long as x = y, we havea 
solution. We can picture all of these solutions by thinking of the graph of the equation 
y = x on the traditional x, y coordinate plane. 

Let’s continue this visual aspect of considering solutions to linear systems. Con- 
sider the system 


x+ty=2 
x—y=0. 


Each of these equations can be viewed as lines in the coordinate plane, and since their 
slopes are different, we know they will intersect somewhere (see Figure 1.1 (a)). In 
this example, they intersect at the point (1,1) — that is, when x = 1 and y = 1, both 
equations are satisfied and we have a solution to our linear system. Since this is the 
only place the two lines intersect, this is the only solution. 

Now consider the linear system 


x+ty=1 
2x +2y = 2. 


It is clear that while we have two equations, they are essentially the same equation; 
the second is just a multiple of the first. Therefore, when we graph the two equations, 
we are graphing the same line twice (see Figure 1.1 (b); the thicker line is used to 
represent drawing the line twice). In this case, we have an infinite solution set, just as 
if we only had the one equation x + y = 1. We often write the solution as x = 1—yto 
demonstrate that y can be any real number, and x is determined once we pick a value 
for y. 
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(a) (b) (c) 


Figure 1.1: The three possibilities for two linear equations with two unknowns. 


Finally, consider the linear system 


x+y=1 
x+y=2. 


We should immediately spot a problem with this system; if the sum of x and y is 1, 
how can it also be 2? There is no solution to such a problem; this linear system has no 
solution. We can visualize this situation in Figure 1.1 (c); the two lines are parallel and 
never intersect. 


If we were to consider a linear system with three equations and two unknowns, we 
could visualize the solution by graphing the corresponding three lines. We can picture 
that perhaps all three lines would meet at one point, giving exactly 1 solution; per- 
haps all three equations describe the same line, giving an infinite number of solutions; 
perhaps we have different lines, but they do not all meet at the same point, giving 
no solution. We further visualize similar situations with, say, 20 equations with two 
variables. 

While it becomes harder to visualize when we add variables, no matter how many 
equations and variables we have, solutions to linear equations always come in one of 
three forms: exactly one solution, infinitely many solutions, or no solution. This is a 
fact that we will not prove just now, but it deserves to be stated. 


Theorem 1 Solution Forms of Linear Systems 


Every linear system of equations has exactly one solution, 
infinitely many solutions, or no solution. 


This leads us to a definition. Here we don’t differentiate between having one so- 
lution and infinitely many solutions, but rather just whether or not a solution exists. 


1.4 Existence and Uniqueness of Solutions 


Definition 5 Consistent and Inconsistent Linear Systems 


A system of linear equations is consistent if it has a solution 
(perhaps more than one). A linear system is inconsistent if 
it does not have a solution. 


How can we tell what kind of solution (if one exists) a given system of linear equa- 
tions has? The answer to this question lies with properly understanding the row ech- 
elon form or reduced row echelon form of a matrix. To discover what the solution is 
to a linear system, we first put the matrix into either of these forms and then interpret 
that form properly. Before we start with a simple example, let us make a note about 
performing these computations. 


Technology Note: In the previous section, we learned how to find the row echelon 
form or reduced row echelon form of a matrix using Gaussian elimination — by hand. 
We need to know how to do this; understanding the process has benefits. However, 
actually executing the process by hand for every problem is not usually beneficial. In 
fact, with large systems, performing these computations by hand is effectively impos- 
sible. Our main concern is these forms are for a given matrix, and what they tell us, 
not what exact steps were used to arrive there. Therefore, the reader is encouraged 
to employ some form of technology to find the row echelon or reduced row echelon 
formof a matrix. Computer programs such as Mathematica, MATLAB, Maple, and De- 
rive can be used; many handheld calculators (such as Texas Instruments calculators) 
will perform these calculations very quickly. 

As a general rule, when we are learning a new technique, it is best to not use 
technology to aid us. This helps us learn not only the technique but some of its “inner 
workings.” We can then use technology once we have mastered the technique and are 
now learning how to use it to solve problems. 

From here on out in our examples we will generally use the reduced row echelon 
form of the system to draw conclusions about solvability, but the conclusions can also 
be drawn by examining the row echelon form. When we need the either the row eche- 
lon form or reduced row echelon form of a matrix, we will not show the steps involved. 
Rather, we will give the initial matrix, then immediately give whichever reduced form 
of the matrix we desire. We trust that the reader can verify the accuracy of this form 
by both performing the necessary steps by hand or utilizing some technology to do it 
for them. 


Our first example explores officially a quick example used in the introduction of 
this section. 


Example 8 Find the solution to the linear system 
Xi + X% = 1 
2X1 + 2x. = 2° 
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SOLUTION Create the corresponding augmented matrix, and then put the ma- 
trix into row echelon form (same as reduced row echelon form here). 


111 -= 1 1 1 
22 2 me 000 


Now convert this matrix back into equations. In this case, we only have one equa- 


tion, 
Xytx%=1 
or, equivalently, 
xX, =1-x 
Xa is free. 


We have just introduced a new term, the word free. It is used to stress that idea 
that x can take on any value; we are “free” to choose any value for x2. Once this value 
is chosen, the value of x; is determined. We have infinitely many choices for the value 
of x2, so therefore we have infinitely many solutions. For example, if we set x = 0, 
then x, = 1; if we set x2 = 5, then x, = —4. 


Let’s try another example, one that uses more variables. 


Example 9 Find the solution to the linear system 
X2 = X3 = 3 
X41 + 2x3 = 2. 
—3x2 =e 3x3 = -9 
SOLUTION To find the solution, put the corresponding matrix into reduced 


row echelon form. 


= 10 2 2 
1 O 2 2 rref 0 1 -1 3 
0 -3 3 -9 00 0 0 


Now convert this reduced matrix back into equations. We have 


Xı + 2x3 =2 


X2 — X3 = 3 
or, equivalently, 


Xı = 2 — 2x3 
Xx =3+x3 


X3 is free. 
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These two equations tell us that the values of x; and x depend on what x3 is. As we 
saw before, there is no restriction on what x3 must be; it is “free” to take on the value 
of any real number. Once x3 is chosen, we have a solution. Since we have infinitely 
many choices for the value of x3, we have infinitely many solutions. 


As examples, x; = 2, X2 = 3, x3 = Ois one solution; x; = —2, x2 = 5, x3 = 2 is 
another solution. Try plugging these values back into the original equations to verify 
that these indeed are solutions. (By the way, since infinitely many solutions exist, this 
system of equations is consistent.) 


In the two previous examples we have used the word “free” to describe certain 
variables. What exactly is a free variable? How do we recognize which variables are 
free and which are not? 


Look back to the reduced matrix in Example 8. Notice that there is only one pivot 
in that matrix, and that pivot corresponded to the x, variable. That told us that x; was 
not a free variable; since x2 did not correspond to a pivot, it was a free variable. 


Look also at the reduced matrix in Example 9. There were two pivots in that matrix; 
one corresponded to x, and the other to x2. This meant that x; and x, were not free 
variables; since there was not a pivot that corresponded to x3, it was a free variable. 


We formally define this and a few other terms in this following definition. 


Definition 6 Dependent and Independent Variables 


Consider the reduced row echelon form of an augmented 
matrix of a linear system of equations. Then: 


a variable that corresponds to a pivot is a basic, or depen- 
dent, variable, and 


a variable that does not correspond to a pivot is a free, or 
independent, variable. 


One can probably see that “free” and “independent” are relatively synonymous. It 
follows that if a variable is not independent, it must be dependent; the word “basic” 
comes from connections to other areas of mathematics that we won’t explore here. 


These definitions help us understand when a consistent system of linear equations 
will have infinitely many solutions. If there are no free variables, then there is exactly 
one solution; if there are any free variables, there are infinitely many solutions. 
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Key Idea 2 Consistent Solution Types 
A consistent linear system of equations will have exactly 
one solution if and only if there is a pivot for each variable 


in the system. 


If a consistent linear system of equations has a free variable, 


it has infinitely many solutions. 


If a consistent linear system has more variables than pivots, 
then the system will have infinitely many solutions. 


A consistent linear system with more variables than equa- 
tions will always have infinitely many solutions. 


Note: Key Idea 2 applies only to consistent systems. If a system is inconsistent, 
then no solution exists and talking about free and basic variables is meaningless. 


When a consistent system has only one solution, each equation that comes from 
the reduced row echelon form of the corresponding augmented matrix will contain 
exactly one variable. If the consistent system has infinitely many solutions, then there 
will be at least one equation coming from the reduced row echelon form that contains 
more than one variable. The “first” variable will be the basic (or dependent) variable; 
all others will be free variables. 

We have now seen examples of consistent systems with exactly one solution and 
others with infinitely many solutions. How will we recognize that a system is inconsis- 
tent? Let’s find out through an example. 


Example 10 Find the solution to the linear system 
Xi + Xx + X% = 1 
X + 2X + X% = 2. 
2X1 +r 3x2 + 2x3 = 0 
SOLUTION We start by putting the corresponding matrix into reduced row 


echelon form. 


NPR 
WNP 
NPR 
ONPR 

“s 

-y 

o 

D: 
OOP 
Oro 
CoOOR 
eOO 


Now let us take the reduced matrix and write out the corresponding equations. 
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The first two rows give us the equations 


Xı +X% =0 


X = 0. 


So far, so good. However the last row gives us the equation 
Ox, + 0x2 + Ox3 = 1 


or, more concisely, O = 1. Obviously, this is not true; we have reached a contradiction. 
Therefore, no solution exists; this system is inconsistent. 


In previous sections we have only encountered linear systems with unique solu- 
tions (exactly one solution). Now we have seen three more examples with different 
solution types. The first two examples in this section had infinitely many solutions, 
and the third had no solution. How can we tell if a system is inconsistent? 

A linear system will be inconsistent only when it implies that O equals 1. We can 
tell if a linear system implies this by putting its corresponding augmented matrix into 
reduced row echelon form. If we have any row where all entries are O except for the 
entry in the last column, then the system implies O=1. More succinctly, if we have a 
pivot in the last column of an augmented matrix, then the linear system has no solu- 
tion. 


Key Idea 3 Inconsistent Systems of Linear Equations 


A system of linear equations is inconsistent if the reduced 
row echelon form of its corresponding augmented matrix 
has a pivot in the last column. 


Example 11 Confirm that the linear system 
x + y = 0 
2x + 2y = 4 


has no solution. 


SOLUTION We can verify that this system has no solution in two ways. First, 
let’s just think about it. If x+y = 0, then it stands to reason, by multiplying both sides 
of this equation by 2, that 2x + 2y = 0. However, the second equation of our system 
says that 2x + 2y = 4. Since 0 Æ 4, we have a contradiction and hence our system has 
no solution. (We cannot possibly pick values for x and y so that 2x + 2y equals both 0 
and 4.) 
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Now let us confirm this using the prescribed technique from above. The reduced 
row echelon form of the corresponding augmented matrix is 


1 1 0 
0 0 14° 
We have a pivot in the last column, so therefore the system is inconsistent. 


Let’s summarize what we have learned up to this point. Consider the reduced row 
echelon form of the augmented matrix of a system of linear equations. If there is 
a pivot in the last column, the system has no solution. Otherwise, if there is a pivot 
for each variable, then there is exactly one solution; otherwise (i.e., there are free 
variables) there are infinitely many solutions. 

Systems with exactly one solution or no solution are the easiest to deal with; sys- 
tems with infinitely many solutions are a bit harder to deal with. Therefore, we’ll doa 
little more practice. First, a definition: if there are infinitely many solutions, what do 
we Call one of those infinitely many solutions? 


Definition 7 Particular Solution 


Consider a linear system of equations with infinitely many 
solutions. A particular solution is one solution out of the 
infinite set of possible solutions. 


The easiest way to find a particular solution is to pick values for the free variables 
which then determines the values of the dependent variables. Again, more practice is 
called for. 


Example 12 Give the solution to a linear system whose augmented matrix in 
reduced row echelon form is 


1-1 0 2 4 
0 0 1 -3 7 
0 0 0 0 0 
and give two particular solutions. 
SOLUTION We can essentially ignore the third row; it does not divulge any 


information about the solution.? The first and second rows can be rewritten as the 


2That sure seems like a mouthful in and of itself. However, it boils down to “look at the reduced form of 
the usual matrix.” 

3Then why include it? Rows of zeros sometimes appear “unexpectedly” in matrices after they have been 
put in reduced row echelon form. When this happens, we do learn something; it means that at least one 
equation was a combination of some of the others. 
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following equations: 


Xı— X + 2x, = 4 


X3 — 3x4 = 7. 


Notice how the variables x; and x3 correspond to the pivots of the given matrix. There- 
fore x; and x3 are dependent variables; all other variables (in this case, x2 and x4) are 
free variables. 

We generally write our solution with the dependent variables on the left and inde- 
pendent variables and constants on the right. It is also a good practice to acknowledge 
the fact that our free variables are, in fact, free. So our final solution would look some- 
thing like 


Xy = 44+ x2 — 2x4 


X2 is free 


X3 = 7 + 3x4 
X4 is free. 
To find particular solutions, choose values for our free variables. There is no “right” 


way of doing this; we are “free” to choose whatever we wish. 
By setting x2 = 0 = x4, we have the solution x, 4, x2 0, x3 7, Xa 0. 


By setting x2 = 1 and x, = —5, we have the solution x; = 15, x2 = 1, x3 = —8, 
X4 = —5. It is easier to read this when are variables are listed vertically, so we repeat 
these solutions: 
One particular solution is: Another particular solution is: 
x= 4 Xı = 15 
X2 = (0) X2: = 1 
X3 = 7 X3 = —8 
X4 = 0. X4 = —5. 
Example 13 Find the solution to a linear system whose augmented matrix in 


reduced row echelon form is 
1002 3 
0104 5 
and give two particular solutions. 


SOLUTION Converting the two rows into equations we have 


Xı + 2x, =3 


X2 + 4x, = 5. 
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We see that x; and x are our dependent variables, for they correspond to the 
pivots. Therefore, x and x, are independent variables. This situation feels a little 
unusual,‘ for x3 doesn’t appear in any of the equations above, but cannot overlook it; 
it is still a free variable since there is not a pivot that corresponds to it. We write our 
solution as: 


Xı = 3- 2X4 
X = 5 — 4x4 
X3 is free 


X4 is free. 


To find two particular solutions, we pick values for our free variables. Again, there 
is no “right” way of doing this (in fact, there are . . . infinitely many ways of doing this) 
so we give only an example here. 


One particular solution is: Another particular solution is: 
x, =3 xX, =3-27 
x%=5 X =5— 4r 
x3 = 1000 x3 =e? 

X4, = 0. X4= 7. 


|’ 


(In the second particular solution we picked “unusua 
light the fact that we can.) 


values for x3 and x, just to high- 


Example 14 Find the solution to the linear system 
X1 + X2 + X3 = 5 
Xt — X22 tr Xe = 3 


and give two particular solutions. 


SOLUTION The corresponding augmented matrix and its reduced row echelon 
form are given below. 


1 1 15 =i 1014 
1 -1 1 3 is 0101 


Converting these two rows into equations, we have 


xX, +%x%3=4 


xX=1 


4What kind of situation would lead to a column of all zeros? To have such a column, the original ma- 
trix needed to have a column of all zeros, meaning that while we acknowledged the existence of a certain 
variable, we never actually used it in any equation. In practical terms, we could respond by removing the 
corresponding column from the matrix and just keep in mind that variable is free. In very large systems, it 

might be hard to determine whether or not a variable is actually used and one would not worry about it. 
When we learn about eigenvectors and eigenvalues, we will see that under certain circumstances this 
34 situation arises. In those cases we leave the variable in the system just to remind ourselves that it is there. 
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giving us the solution 


X1 =4-x3 
X2 = 1 
X3 is free. 


|’ 


Once again, we get a bit of an “unusual” solution; while x2 is a dependent variable, 
it does not depend on any free variable; instead, it is always 1. (We can think of it 
as depending on the value of 1.) By picking two values for x3, we get two particular 
solutions. 


One particular solution is: Another particular solution is: 
x,y =4 x,=3 
X2 = 1 X2 = 1 
X3 = 0. X3 = 1. 


The constants and coefficients of a matrix work together to determine whether 
a given system of linear equations has one, infinitely many, or no solution. The con- 
cept will be fleshed out more in later chapters, but in short, the coefficients determine 
whether a matrix will have exactly one solution or not. In the “or not” case, the con- 
stants determine whether or not infinitely many solutions or no solution exists. (So if 
a given linear system has exactly one solution, it will always have exactly one solution 
even if the constants are changed.) Let’s look at an example to get an idea of how the 
values of constants and coefficients work together to determine the solution type. 


Example 15 For what values of k will the given system have exactly one solu- 
tion, infinitely many solutions, or no solution? 


X% + 2x = 3 
3x, + kx» = 9 
SOLUTION We answer this question by forming the augmented matrix and 


starting the process of putting it into reduced row echelon form. Below we see the 
augmented matrix and one elementary row operation that starts the Gaussian elimi- 
nation process. 


tasj => fi = 3 
3 k | Bu ae E k—6 i 


This is as far as we need to go. In looking at the second row, we see that if k = 6, 
then that row contains only zeros and x is a free variable; we have infinitely many 
solutions. If k Æ 6, then our next step would be to make that second row, second 
column entry a leading one. We don’t particularly care about the solution, only that 
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we would have exactly one as both x; and x would correspond to a leading one and 
hence be dependent variables. 

Our final analysis is then this. If k 4 6, there is exactly one solution; if k = 6, there 
are infinitely many solutions. In this example, it is not possible to have no solutions. 


As an extension of the previous example, consider the similar augmented matrix 
where the constant 9 is replaced with a 10. Performing the same elementary row 
operation gives 


12 3 ee ee ae 
E k A SE a p k—6 ile 


As in the previous example, if k 4 6, we can make the second row, second column 
entry a leading one and hence we have one solution. However, if k = 6, then our last 
row is [0 0 1], meaning we have no solution. 


It should be clear from the above examples that the pivots in the reduced echelon 
form play an important role in the nature of the solutions in a consistent system.In 
particular, the number of pivots dictates the number of free and basic variables. This 
motivates a definition. 


Definition 8 Rank of a Matrix 


The rank of a matrix A is the number of pivots in the reduced 
row echelon form of A. 


The rank of a matrix A is usually denoted by rank(A). A few observations are in 
order. Let A be an m x n matrix: 


1. rank(A) < min(m,n) (since there is at most one pivot element per row or col- 
umn of A). 


2. The number of basic variables in a consistent system Ax = b equals rank(A). 
3. The number of free variables in a consistent system Ax = b equals n — rank(A). 


When rank(A) equals m, the number of rows, we say that A has full row rank; similarly, 
when rank(A) equals n, the number of columns, we say that A has full column rank. If 
Ais a square n x n matrix of rank n we say simply that A has full rank. Thus if a matrix 
A has full column rank and Ax = b is consistent, the solution is unique since there are 
no free variables. It’s also easy to see that if A has full row rank the system Ax = bis 
consistent for any b, since we can never arrive at a row of the form [00 --- O 1ļin 
the augmented matrix for AX = b. 

For example, in Example 12 the matrix A (which corresponds to the four leftmost 
columns) has rank 2. There are thus 2 basic variables, and 4 — 2 = 2 free variables. 
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In Example 13 the matrix A (which corresponds to the four leftmost columns) also has 
rank 2. In this case there are also 2 basic variables, and 4 — 2 = 2 free variables. In 
Example 14 the matrix A (which corresponds to the three leftmost columns) has rank 
2. There are thus 2 basic variables, and 3 — 2 = 1 free variables. 

We have been studying the solutions to linear systems mostly in an “academic” 
setting; we have been solving systems for the sake of solving systems. In the next sec- 
tion, we’ll look at situations which create linear systems that need solving (i.e., “word 
problems”). 


Exercises 1.4 


In Exercises 1 — 14, find the solution to the 2X1 + xQ + 2x3 = 0 
given linear system. If the system has infinite 1. x + OX + 3% = 1 
solutions, give 2 particular solutions. 3x1 2x2 5x% = 3 
1 2x1 + 4%. = 2 Xı + 32 + 3x3 = 1 
mM + 22 = 1 12. 2% — x. + 2x3 = -1 
7 -x ss 5x2 = 3 4x. 1 + 5x: 2 +r 8x: 3 = 2 
2X1 -= 10x2 = —6 xı 2x2 2x3 = 1 
xX, +x = 3 13. 2x + X2 + 3x3 = 1 
3. 2x1 + x = 4 3x1 + 3x2 + 5x3 = 2 
4 —3x, + 7x2 = -7 2x. + 4x. + 6x3 = 2 
` 2X1 — 8) = 8 14. 1x1 2X2 3x3 = 1 
—3X1 = 6x2 = 9x3 = —3 
5 2X1 + 3x2 = 1 
' —2x%x — 3x. = 1 In Exercises 15 — 18, state for which values 
of k the given system will have exactly 1 solu- 
6. X t x =l tion, infinite solutions, or no solution. 
—X1 == 2x2 = 5 
X1 + 2X2 = 1 
— + = 15. 
1 2a t Mo 4x3 6 Oy + Weg = k 
X1 _ 3x2 +r 2X3 = 1 
g wa + w + We = 2 iG 2 = ee = 
© 2x + 5a + x = 2 ae 
X1 —X2 +X3 +X, = 0 17 Xi + 2x2 = 1 
9. A 
2X1 2X2 +X% = 1 Xx + ka = 2 
Xt +X + 6x3 + 9X4 = 0 X1 + 2X2 = 1 
10. 18. 
Xı X3 2X4 = 3 X1 + 3x2 = k 
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AS YOU READ 


1. How do most problems appear “in the real world?” 
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2. The unknowns in a problem are also called what? 


3. How many points are needed to determine the coefficients of a 5* degree poly- 
nomial? 


We’ve started this chapter by addressing the issue of finding the solution to a sys- 
tem of linear equations. In subsequent sections, we defined matrices to store linear 
equation information; we described how we can manipulate matrices without chang- 
ing the solutions; we described how to efficiently manipulate matrices so that a work- 
ing solution can be easily found. 

We shouldn’t lose sight of the fact that our work in the previous sections was aimed 
at finding solutions to systems of linear equations. In this section, we’ll learn how to 
apply what we’ve learned to actually solve some problems. 

Many, many, many problems that are addressed by engineers, businesspeople, 
scientists and mathematicians can be solved by properly setting up systems of linear 
equations. In this section we highlight only a few of the wide variety of problems that 
matrix algebra can help us solve. 

We start with a simple example. 


Example 16 A jar contains 100 blue, green, red and yellow marbles. There are 
twice as many yellow marbles as blue; there are 10 more blue marbles than red; the 
sum of the red and yellow marbles is the same as the sum of the blue and green. How 
many marbles of each color are there? 


SOLUTION Let’s call the number of blue balls b, and the number of the other 
balls g, rand y, each representing the obvious. Since we know that we have 100 mar- 
bles, we have the equation 

b+g+r+y= 100. 


The next sentence in our problem statement allows us to create three more equations. 
We are told that there are twice as many yellow marbles as blue. One of the fol- 
lowing two equations is correct, based on this statement; which one is it? 


2y=b or 2b=y 


The first equation says that if we take the number of yellow marbles, then double 
it, we’ll have the number of blue marbles. That is not what we were told. The second 
equation states that if we take the number of blue marbles, then double it, we’ll have 
the number of yellow marbles. This is what we were told. 

The next statement of “there are 10 more blue marbles as red” can be written as 
either 

b=r+10 or r=b +10. 
Which is it? 
The first equation says that if we take the number of red marbles, then add 10, 


we'll have the number of blue marbles. This is what we were told. The next equation 
is wrong; it implies there are more red marbles than blue. 
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The final statement tells us that the sum of the red and yellow marbles is the same 
as the sum of the blue and green marbles, giving us the equation 
r+y=b+g. 


We have four equations; altogether, they are 


b+g+r+y= 100 


2b = y 
b=r+10 
r+y=b+g. 


We want to write these equations in a standard way, with all the unknowns on the 
left and the constants on the right. Let us also write them so that the variables appear 
in the same order in each equation (we’ll use alphabetical order to make it simple). 
We now have 


b+g+r+y= 100 
2b—y=0 
b-—r=10 
—b-—g+r+y=0 


To find the solution, let’s form the appropriate augmented matrix and put it into 
reduced row echelon form. We do so here, without showing the steps. 


1 1 1 1 100 100 0 20 
2 0 0 -1 0 010 0 30 
1 0 -1 0 10 rrer 001010 
-1 -1 1 1 0 0001 40 


We interpret from the reduced row echelon form of the matrix that we have 20 
blue, 30 green, 10 red and 40 yellow marbles. 


Even if you had a bit of difficulty with the previous example, in reality, this type 
of problem is pretty simple. The unknowns were easy to identify, the equations were 
pretty straightforward to write (maybe a bit tricky for some), and only the necessary 
information was given. 

Most problems that we face in the world do not approach us in this way; most 
problems do not approach us in the form of “Here is an equation. Solve it.” Rather, 
most problems come in the form of: 


Here is a problem. | want the solution. To help, here is lots of information. 
It may be just enough; it may be too much; it may not be enough. You 
figure out what you need; just give me the solution. 
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Faced with this type of problem, how do we proceed? Like much of what we’ve 
done in the past, there isn’t just one “right” way. However, there are a few steps 
that can guide us. You don’t have to follow these steps, “step by step,” but if you find 
that you are having difficulty solving a problem, working through these steps may help. 
(Note: while the principles outlined here will help one solve any type of problem, these 
steps are written specifically for solving problems that involve only linear equations.) 


Key Idea 4 Mathematical Problem Solving 


1. Understand the problem. What exactly is being 
asked? 


. Identify the unknowns. What are you trying to find? 
What units are involved? 


. Give names to your unknowns (these are your vari- 
ables). 


. Use the information given to write as many equations 
as you can that involve these variables. 


. Use the equations to form an augmented matrix; use 
Gaussian elimination to put the matrix into reduced 
row echelon form. 


. Interpret the reduced row echelon form of the matrix 
to identify the solution. 


. Ensure the solution makes sense in the context of the 
problem. 


Having identified some steps, let us put them into practice with some examples. 


Example 17 A concert hall has seating arranged in three sections. As part of a 
special promotion, guests will recieve two of three prizes. Guests seated in the first 
and second sections will receive Prize A, guests seated in the second and third sections 
will receive Prize B, and guests seated in the first and third sections will receive Prize 
C. Concert promoters told the concert hall managers of their plans, and asked how 
many seats were in each section. (The promoters want to store prizes for each section 
separately for easier distribution.) The managers, thinking they were being helpful, 
told the promoters they would need 105 A prizes, 103 B prizes, and 88 C prizes, and 
have since been unavailable for further help. How many seats are in each section? 


SOLUTION Before we rush in and start making equations, we should be clear 
about what is being asked. The final sentence asks: “How many seats are in each 
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section?” This tells us what our unknowns should be: we should name our unknowns 
for the number of seats in each section. Let x1, X2 and x3 denote the number of seats 
in the first, second and third sections, respectively. This covers the first two steps of 
our general problem solving technique. 

(It is tempting, perhaps, to name our variables for the number of prizes given away. 
However, when we think more about this, we realize that we already know this — that 
information is given to us. Rather, we should name our variables for the things we 
don’t know.) 

Having our unknowns identified and variables named, we now proceed to forming 
equations from the information given. Knowing that Prize A goes to guests in the first 
and second sections and that we’ll need 105 of these prizes tells us 


Xı +X = 105. 
Proceeding in a similar fashion, we get two more equations, 
X2 +X3 = 103 and Xı +X3 = 88. 


Thus our linear system is 


Xi +X. = 105 
X2 TX = 103 
Xi +xX3 = 88 


and the corresponding augmented matrix is 


1 1 0 105 = 1 0 0 45 
O 1 1 103 rref O 1 0 60 
101 88 00 1 43 
We can now read off our solution. The first section has 45 seats, the second has 


60 seats, and the third has 43 seats. 


Example 18 A women takes a 2-mile motorized boat trip down the Highwater 
River, knowing the trip will take 30 minutes. She asks the boat pilot “How fast does 
this river flow?” He replies “I have no idea. | just drive the boat.” 

She thinks for a moment, then asks “How long does the return trip take?” He 
replies “The same; half an hour.” She follows up with the statement, “Since both legs 
take the same time, you must not drive the boat at the same speed.” 

“Naw,” the pilot said. “While | really don’t know exactly how fast | go, | do know 
that since we don’t carry any tourists, | drive the boat twice as fast.” 

The woman walks away satisfied; she knows how fast the river flows. 

(How fast does it flow?) 
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SOLUTION This problem forces us to think about what information is given 
and how to use it to find what we want to know. In fact, to find the solution, we'll find 
out extra information that we weren’t asked for! 

We are asked to find how fast the river is moving (step 1). To find this, we should 
recognize that, in some sense, there are three speeds at work in the boat trips: the 
speed of the river (which we want to find), the speed of the boat, and the speed that 
they actually travel at. 

We know that each leg of the trip takes half an hour; if it takes half an hour to cover 
2 miles, then they must be traveling at 4 mph, each way. 

The other two speeds are unknowns, but they are related to the overall speeds. 
Let’s call the speed of the river r and the speed of the boat b. (And we should be 
careful. From the conversation, we know that the boat travels at two different speeds. 
So we’ll say that b represents the speed of the boat when it travels downstream, so 
2b represents the speed of the boat when it travels upstream.) Let’s let our speed be 
measured in the units of miles/hour (mph) as we used above (steps 2 and 3). 

What is the rate of the people on the boat? When they are travelling downstream, 
their rate is the sum of the water speed and the boat speed. Since their overall speed 
is 4 mph, we have the equationr + b = 4. 

When the boat returns going against the current, its overall speed is the rate of 
the boat minus the rate of the river (since the river is working against the boat). The 
overall trip is still taken at 4 mph, so we have the equation 2b — r = 4. (Recall: the 
boat is traveling twice as fast as before.) 

The corresponding augmented matrix is 


1 1 4 
2 -1 4] 
Note that we decided to let the first column hold the coefficients of b. 
Putting this matrix in reduced row echelon form gives us: 
1 1 4 = 1 0 8/3 
2-1 4 0 1 4/33’ 
We finish by interpreting this solution: the speed of the boat (going downstream) 


is 8/3 mph, or 2.6 mph, and the speed of the river is 4/3 mph, or 1.3 mph. All we really 
wanted to know was the speed of the river, at about 1.3 mph. 


Example 19 Find the equation of the quadratic function that goes through the 
points (—1,6), (1,2) and (2, 3). 


SOLUTION This may not seem like a “linear” problem since we are talking 
about a quadratic function, but closer examination will show that it really is. 

We normally write quadratic functions as y = ax? + bx + c where a, b and c are 
the coefficients; in this case, they are our unknowns. We have three points; consider 
the point (—1, 6). This tells us directly that if x = —1, then y = 6. Therefore we know 
that 6 = a(—1)?+b6(—1) +c. Writing this in a more standard form, we have the linear 
equation 

a—b+c=6. 


1.5 Applications of Linear Systems 


The second point tells us that a(1)? + b(1) +c = 2, which we can simplify as 
a +b +c = 2, and the last point tells us a(2)? + b(2) +c = 3, or 4a + 2b + c = 3. 
Thus our linear system is 


a—b+c = 6 
a+b+c = 2 
4a+2b+c = 3. 


Again, to solve our system, we find the reduced row echelon form of the corre- 
sponding augmented matrix. We don’t show the steps here, just the final result. 


1 -1 1 6 100 1 
1 1 1 2 rref 0 1 0 -2 
4 2 1 3 001 3 
This tells us thata = 1, b = —2 and c = 3, giving us the quadratic function 


y=x*—2x+4+3. 


One thing interesting about the previous example is that it confirms for us some- 
thing that we may have known for a while (but didn’t know why it was true). Why do 
we need two points to find the equation of the line? Because in the equation of the 
a line, we have two unknowns, and hence we’ll need two equations to find values for 
these unknowns. 

A quadratic has three unknowns (the coefficients of the x? term and the x term, and 
the constant). Therefore we’ll need three equations, and therefore we’ll need three 
points. 

What happens if we try to find the quadratic function that goes through 3 points 
that are all on the same line? The fast answer is that you’ll get the equation of a line; 
there isn’t a quadratic function that goes through 3 colinear points. Try it and see! 
(Pick easy points, like (0,0), (1,1) and (2,2). You'll find that the coefficient of the x? 
term is 0.) 

Of course, we can do the same type of thing to find polynomials that go through 4, 
5, etc., points. In general, if you are given n+ 1 points, a polynomial that goes through 
alln + 1 points will have degree at most n. 


Example 20 Awoman has 32 $1, $5 and $10 bills in her purse, giving her a total 
of $100. How many bills of each denomination does she have? 


SOLUTION Let’s name our unknowns x, y and z for our ones, fives and tens, 
respectively (it is tempting to call them o, f and t, but o looks too much like 0). We 
know that there are a total of 32 bills, so we have the equation 


xXt+ty+z= 32. 
We also know that we have $100, so we have the equation 
x + 5y + 10z = 100. 


We have three unknowns but only two equations, so we know that we cannot expect 
a unique solution. Let’s try to solve this system anyway and see what we get. 
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Putting the system into a matrix and then finding the reduced row echelon form, 
we have 


1 1 1 32 —> 1 0 —-} 15 
1 5 10 100 


Reading from our reduced matrix, we have the infinite solution set 


O 5 

=lat Z 

9 

y=17- z 
zis free. 


While we do have infinite solutions, most of these solutions really don’t make sense 
in the context of this problem. (Setting z = ł doesn’t make sense, for having half a 
ten dollar bill doesn’t give us $5. Likewise, having z = 8 doesn’t make sense, for then 
we'd have “—1” $5 bills.) So we must make sure that our choice of z doesn’t give us 
fractions of bills or negative amounts of bills. 

To avoid fractions, z must be a multiple of 4 (—4, 0, 4,8,...). Of course, z > O for 
a negative number wouldn’t make sense. If z = 0, then we have 15 one dollar bills 
and 17 five dollar bills, giving us $100. If z = 4, then we have x = 20 andy = 8. 
We already mentioned that z = 8 doesn’t make sense, nor does any value of z where 
z> 8. 

So it seems that we have two answers; one with z = 0 and one with z = 4. Of 
course, by the statement of the problem, we are led to believe that the woman has 
at least one $10 bill, so probably the “best” answer is that we have 20 $1 bills, 8 $5 
bills and 4 $10 bills. The real point of this example, though, is to address how infinite 
solutions may appear in a real world situation, and how suprising things may result. 


Example 21 In a football game, teams can score points through touchdowns 
worth 6 points, extra points (that follow touchdowns) worth 1 point, two point con- 
versions (that also follow touchdowns) worth 2 points and field goals, worth 3 points. 
You are told that in a football game, the two competing teams scored on 7 occasions, 
giving a total score of 24 points. Each touchdown was followed by either a successful 
extra point or two point conversion. In what ways were these points scored? 


SOLUTION The question asks how the points were scored; we can interpret 
this as asking how many touchdowns, extra points, two point conversions and field 
goals were scored. We’ll need to assign variable names to our unknowns; let t rep- 
resent the number of touchdowns scored; let x represent the number of extra points 
scored, let w represent the number of two point conversions, and let f represent the 
number of field goals scored. 

Now we address the issue of writing equations with these variables using the given 
information. Since we have a total of 7 scoring occasions, we know that 


t+x+w+f=/7. 


1.5 Applications of Linear Systems 
The total points scored is 24; considering the value of each type of scoring opportunity, 
we can write the equation 
6t +x + 2w+ 3f = 24. 


Finally, we know that each touchdown was followed by a successful extra point or two 
point conversion. This is subtle, but it tells us that the number of touchdowns is equal 
to the sum of extra points and two point conversions. In other words, 


t=x-+w. 


To solve our problem, we put these equations into a matrix and put the matrix into 
reduced row echelon form. Doing so, we find 


1 1 1 1 7 = 1 0 0 05 3.5 
6 1 2 3 24 rref 01 0 1 4 
1 -1 -1 0 0 001 —0.5 —0.5 


Therefore, we know that 


t= 3.5 — 0.5f 
x=4-f 
w = —0.5 + 0.5f. 


We recognize that this means there are “infinite solutions,” but of course most of these 
will not make sense in the context of a real football game. We must apply some logic 
to make sense of the situation. 

Progressing in no particular order, consider the second equation, x = 4 — f. In 
order for us to have a positive number of extra points, we must have f < 4. (And of 
course, we need f > 0, too.) Therefore, right away we know we have a total of only 5 
possibilities, where f = O, 1, 2, 3 or 4. 

From the first and third equations, we see that if fis an even number, then t and w 
will both be fractions (for instance, if f = 0, then t = 3.5) which does not make sense. 
Therefore, we are down to two possible solutions, f = 1and f = 3. 

If f = 1, we have 3 touchdowns, 3 extra points, no two point conversions, and 
(of course), 1 field goal. (Check to make sure that gives 24 points!) If f = 3, then 
we 2 touchdowns, 1 extra point, 1 two point conversion, and (of course) 3 field goals. 
Again, check to make sure this gives us 24 points. Also, we should check each solution 
to make sure that we have a total of 7 scoring occasions and that each touchdown 
could be followed by an extra point or a two point conversion. 


We have seen a variety of applications of systems of linear equations. We would 
do well to remind ourselves of the ways in which solutions to linear systems come: 
there can be exactly one solution, infinite solutions, or no solutions. While we did see 
a few examples where it seemed like we had only 2 solutions, this was because we 
were restricting our solutions to “make sense” within a certain context. 
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We should also remind ourselves that linear equations are immensely important. 
The examples we considered here ask fundamentally simple questions like “How fast is 
the water moving?” or “What is the quadratic function that goes through these three 
points?” or “How were points in a football game scored?” The real “important” situ- 
ations ask much more difficult questions that often require thousands of equations! 
(Gauss began the systematic study of solving systems of linear equations while trying 
to predict the next sighting of a comet; he needed to solve a system of linear equations 
that had 17 unknowns. Today, this a relatively easy situation to handle with the help of 
computers, but to do it by hand is a real pain.) Once we understand the fundamentals 
of solving systems of equations, we can move on to looking at solving bigger systems 
of equations; this text focuses on getting us to understand the fundamentals. 


Exercises 1.5 


In Exercises 1 — 5, find the solution of the as red; that the number of blue and yel- 
given problem by: low marbles together is the same as the 
number of green; and that three times 
the number of yellow marbles together 
with the red marbles gives the same 
numbers as the blue marbles. How 
many of each color of marble are in the 
(c) putting the augmented matrix into re- jar? 
duced row echelon form 


(a) creating an appropriate system of linear 
equations 


(b) forming the augmented matrix that cor- 
responds to this system 


5. Arescue mission has 85 sandwiches, 65 


(d) interpreting the reduced row echelon bags of chips and 210 cookies. They 
form of the matrix as a solution know from experience that men will 
1. A farmer looks out his window at his eat 2 sandwiches, 1 bag of chips and 
chickens and pigs. He tells his daugh- 4 cookies; women will eat 1 sandwich, 
ter that he sees 62 heads and 190 legs. a bag of chips and 2 cookies; kids will 
How many chickens and pigs does the eat half a sandwhich, a bag of chips and 
farmer have? 3 cookies. If they want to use all their 
: food up, how many men, women and 

2. A person buys 20 trinkets at a yard sale. kids can they feed? 


The cost of each trinket is either $0.30 
or $0.65. If this person spends $8.80, !n Exercises 6 — 15, find the polynomial with 


how many of each type of trinket do the smallest degree that goes through the 


they buy? given points. 
3. Acarpenter can make two sizes of table, 6. (1,3) and (3, 15) 

grande and venti. The grande table re- 7. (—2,14) and (3, 4) 

quires 4 table legs and 1 table top; the 

venti requires 6 table legs and 2 table 8. (1,5), (—1,3) and (3, —1) 

tops. After doing work, he counts up 9. (—4, —3), (0,1) and (1, 4.5) 

spare parts in his warehouse and real- 

izes that he has 86 table tops left over, 10. (—1, —8), (1, —2) and (3, 4) 

and 300 legs. How many tables of each 11. (—3,3), (1,3) and (2,3) 

kind can he build and use up exactly all 

of his materials? 12. (—2, 15), (—1,4), (1,0) and (2, —5) 
4. A jar contains 100 marbles. We know 13. (—2, —7), (1,2), (2,9) and (3, 28) 

there are twice as many green marbles 14. (—3, 10), (—1, 2), (1,2) and (2,5) 


15. 
16. 


17. 


18. 


(0,1), (—3, —3.5), (—2, —2) and (4, 7) 
The general exponential function has 
the form f(x) = ae™, where a and bare 
constants and e is Euler’s constant (~ 
2.718). We want to find the equation 
of the exponential function that goes 
through the points (1, 2) and (2, 4). 


(a) Show why we cannot simply sub- 
situte in values for x and y in 
y = ae™ and solve using the 
techniques we used for polyno- 
mials. 


(b) Show how the equality y = ae™ 
leads us to the linear equation 
Iny = Ina + bx. 

(c) Use the techniques we devel- 
oped to solve for the unknowns 
Ina and b. 

(d) Knowing Ina, find a; find the ex- 
ponential function f(x) = ae™ 
that goes through the points 
(1, 2) and (2, 4). 


In a football game, 24 points are scored 
from 8 scoring occasions. The number 
of successful extra point kicks is equal 
to the number of successful two point 
conversions. Find all ways in which the 
points may have been scored in this 
game. 


In a football game, 29 points are scored 
from 8 scoring occasions. There are 2 


19. 


20. 


21. 


22. 


23. 


24. 


1.5 Applications of Linear Systems 


more successful extra point kicks than 
successful two point conversions. Find 
all ways in which the points may have 
been scored in this game. 


In a basketball game, where points are 
scored either by a 3 point shot, a 2 point 
shot or a 1 point free throw, 80 points 
were scored from 30 successful shots. 
Find all ways in which the points may 
have been scored in this game. 


In a basketball game, where points are 
scored either by a 3 point shot, a 2 point 
shot or a 1 point free throw, 110 points 
were scored from 70 successful shots. 
Find all ways in which the points may 
have been scored in this game. 


Describe the equations of the linear 
functions that go through the point 
(1,3). Give 2 examples. 


Describe the equations of the linear 
functions that go through the point 
(2,5). Give 2 examples. 


Describe the equations of the quadratic 
functions that go through the points 
(2, —1) and (1,0). Give 2 examples. 


Describe the equations of the quadratic 
functions that go through the points 
(—1, 3) and (2,6). Give 2 examples. 
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MATRIX ARITHMETIC 


A fundamental topic of mathematics is arithmetic; adding, subtracting, multiplying and 
dividing numbers. After learning how to do this, most of us went on to learn how to 
add, subtract, multiply and divide “x”. We are comfortable with expressions such as 


x+3x—x-x*+x°-x7? 


and know that we can “simplify” this to 
4x — x? + x", 


This chapter deals with the idea of doing similar operations, but instead of an un- 
known number x, we will be using a matrix A. So what exactly does the expression 


A+3A—-A-A?+A°-At 


mean? We are going to need to learn to define what matrix addition, scalar multiplica- 
tion, matrix multiplication and matrix inversion are. We will learn just that, plus some 
more good stuff, in this chapter. 


2.1 Matrix Addition and Scalar Multiplication 


AS YOU READ 


1. When are two matrices equal? 


2. Write an explanation of how to add matrices as though writing to someone who 
knows what a matrix is but not much more. 


3. T/F: There is only 1 zero matrix. 


4. T/F: To multiply a matrix by 2 means to multiply each entry in the matrix by 2. 
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In the past, when we dealt with expressions that used “x,” we didn’t just add and 
multiply x’s together for the fun of it, but rather because we were usually given some 
sort of equation that had x in it and we had to “solve for x.” 

This begs the question, “What does it mean to be equal?” Two numbers are equal, 
when, ..., uh, ..., nevermind. What does it mean for two matrices to be equal? We 
say that matrices A and B are equal when their corresponding entries are equal. This 
seems like a very simple definition, but it is rather important, so we give it a box. 


Definition 9 Matrix Equality 


Two m x n matrices A and B are equal if their corresponding 
entries are equal. 


Notice that our more formal definition specifies that if matrices are equal, they 
have the same dimensions. This should make sense. 

Now we move on to describing how to add two matrices together. To start off, take 
a wild stab: what do you think the following sum is equal to? 


alel a]? 
le x) 


you guessed correctly. That wasn’t so hard, was it? 
Let’s keep going, hoping that we are starting to get on a roll. Make another wild 
guess: what do you think the following expression is equal to? 


1 2 
. =? 
ee 


3 6 
eur 
you guessed correctly! 

Even if you guessed wrong both times, you probably have seen enough in these 
two examples to have a fair idea now what matrix addition and scalar multiplication 
are all about. 

Before we formally define how to perform the above operations, let us first recall 
that if A isan m x n matrix, then we can write A as 


If you guessed 


If you guessed 


Q11 Qi2 ++ At 

421 422 ` An 
A = 

Am1 Am2 4 Amn 


2.1 Matrix Addition and Scalar Multiplication 


Secondly, we should define what we mean by the word scalar. A scalar is any number 
that we multiply a matrix by. (In some sense, we use that number to scale the matrix.) 
We are now ready to define our first arithmetic operations. 


Definition 10 Matrix Addition 


Let A and B be m x n matrices. The sum of A and B, denoted 
A+ B, is 


ain +bu ay2+by +++ Gint bin 
aza tba 9 G22 ba = Ayn + ban 


am + bmi Am2 T bm2 ms Amn + bmn 


Definition 11 Scalar Multiplication 


Let A be an m x n matrix and let k be a scalar. The scalar 
multiplication of k and A, denoted kA, is 


kay kay OOS kadin 
kazı kar aoh kazn 


kamı kam2 carn kamn 


We are now ready for an example. 


Example 22 Let 


Simplify the following matrix expressions. 


1. A+B 3. A—B 5. —3A + 2B 7. 5A +5B 
2. B+A 4. A+C 6. A—A 8. 5(A +B) 
SOLUTION 
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oO w 
RO 
w wo 


1.A+B= 


A 
uw 
wo 


O w 
OO 
w wo 


2. B+A= 


A 
uw 
wo 


—-1 -2 -3 
3. A—-B=|-2 0 -1 
6 5 1 


4. A + Cis not defined. If we look at our definition of matrix addition, we see 
that the two matrices need to be the same size. Since A and C have different 
dimensions, we don’t even try to create something as an addition; we simply 
say that the sum is not defined. 


1 2 3 
5. —3A + 2B = 5 —2 1 
—17 —15 -7 


5 10 15 10 20 30 15 30 45 
7. Strictly speaking, thisis | -5 10 5 |+| 5 10 10} =] 0 20 15 
25 25 25 —5 O 20 20 25 45 
8. Strictly speaking, this is 

1 2 3 2 4 6 3 6 9 

5 —1 2 1|+|1 2 2 =5. |0 4 3 

5 5 5 —1 0 4 45 9 

15 30 45 

=]|0O 20 15 

20 25 45 


Our example raised a few interesting points. Notice how A + B = B + A. We 
probably aren’t suprised by this, since we know that when dealing with numbers, a + 
b = b + a. Also, notice that 5A + 5B = 5(A + B). In our example, we were careful to 
compute each of these expressions following the proper order of operations; knowing 
these are equal allows us to compute similar expressions in the most convenient way. 

Another interesting thing that came from our previous example is that 


2.1 Matrix Addition and Scalar Multiplication 


It seems like this should be a special matrix; after all, every entry is O and 0 is a special 
number. 

In fact, this is a special matrix. We define 0, which we read as “the zero matrix,” 
to be the matrix of all zeros.1 We should be careful; this previous “definition” is a bit 
o J 


ambiguous, for we have not stated what size the zero matrix should be. Is 0 0 


the zero matrix? How about [O 0]? 

Let’s not get bogged down in semantics. If we ever see 0 in an expression, we 
will usually know right away what size 0 should be; it will be the size that allows the 
expression to make sense. If A is a3 x 5 matrix, and we write A + 0, we’ll simply 
assume that 0 is also a 3 x 5 matrix. If we are ever in doubt, we can add a subscript; 
for instance, 02x7 is the 2 x 7 matrix of all zeros. 

Since the zero matrix is an important concept, we give it it’s own definition box. 


Definition 12 The Zero Matrix 


The m x n matrix of all zeros, denoted On xn, is the zero 


matrix. 


When the dimensions of the zero matrix are clear from the 
context, the subscript is generally omitted. 


The following presents some of the properties of matrix addition and scalar multi- 
plication that we discovered above, plus a few more. 


Theorem 2 Properties of Matrix Addition and Scalar Multiplication 


The following equalities hold for all m x n matrices A, B and 
C and scalars k. 


. A+B = B + A (Commutative Property) 


. (A+B) +C=A+ (B + C) (Associative Property) 


. k(A +B) = kA + kB (Scalar Multiplication Distributive 
Property) 


. kA = Ak 
. A +0 = 0 + A = A (Additive Identity) 


. 0A=0 


Be sure that this last property makes sense; it says that if we multiply any matrix 
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by the number O, the result is the zero matrix, or 0. 
We began this section with the concept of matrix equality. Let’s put our matrix 
addition properties to use and solve a matrix equation. 


Example 23 Let 


Find the matrix X such that 
2A + 3X = —4A. 


SOLUTION We can use basic algebra techniques to manipulate this equation 
for X; first, let’s subtract 2A from both sides. This gives us 
3X = —6A. 
Now divide both sides by 3 to get 
X = —2A. 


Now we just need to compute —24A; we find that 
—4 2 
x= [3 


Our matrix properties identified 0 as the Additive Identity; i.e., if you add 0 to any 
matrix A, you simply get A. This is similar in notion to the fact that for all numbers a, 
a + 0 = a. A Multiplicative Identity would be a matrix I where I x A = A for all matri- 
ces A. (What would such a matrix look like? A matrix of all 1s, perhaps?) However, in 
order for this to make sense, we’ll need to learn to multiply matrices together, which 
we'll do in the next section. 


Exercises 2.1 


Matrices A and B are given below. In Exer- 5. 3(A—B) +B 


cises 1 — 6, simplify the given expression. 
ii d i 6. 2(A — B) — (A — 3B) 


A= t l B= ee Matrices A and B are given below. In Exer- 
7 4 5 9 . ae : ; 
cises 7 — 10, simplify the given expression. 
3 —2 
1. A+B a=] sal 
2. 2A — 3B 
3. 3A—A 7. 4B — 2A 
4. 4B — 2A 8. —2A + 3A 


9. —2A — 3A 
10. —B + 3B — 2B 


Matrices A and B are given below. In Exer- 
cises 11 — 14, find X that satisfies the equa- 


tion. 
[3 —1 1 7 
AT |2 J s-h; —4 
11. 2A+X=B 
12. A—X=3B 
13. 3A + 2X = —1B 


14. A — 5X = —B 


In Exercises 15 — 21, find values for the scalars 
a and b that satisfy the given equation. 


2.2 Matrix Multiplication 


AS YOU READ 


15. 


16. 


17. 


18. 


19. 


20. 


21. 


2.2 Matrix Multiplication 


eleis] is] 
[eef] i] 
aR 


el-e 
[eleli 
aa 
lell, 


1. T/F: Column vectors are used more in this text than row vectors, although some 


other texts do the opposite. 


2. T/F: To multiply A x B, the number of rows of A and B need to be the same. 


3. T/F: The entry in the 27 row and 3 column of the product AB comes from 
multipling the 2" row of A with the 3 column of B. 


4. Name two properties of matrix multiplication that also hold for “regular multi- 


plication” of numbers. 


5. Name a property of “regular multiplication” of numbers that does not hold for 


matrix multiplication. 


6. T/F:A2 =A-A-A 


In the previous section we found that the definition of matrix addition was very 
intuitive, and we ended that section discussing the fact that eventually we’d like to 
know what it means to multiply matrices together. 

In the spirit of the last section, take another wild stab: what do you think 


[s áll 


1 
2 


2 | 


55 


Chapter 2 Matrix Arithmetic 


56 


means? 
You are likely to have guessed 


se 


but this is, in fact, not right.The actual answer is 


[ns 


If you can look at this one example and suddenly understand exactly how matrix mul- 
tiplication works, then you are probably smarter than the author. While matrix multi- 
plication isn’t hard, it isn’t nearly as intuitive as matrix addition is. 

To further muddy the waters (before we clear them), consider 


1 2 sf 1 -1 0 

3 4 2 2 —1ļ|` 
Our experience from the last section would lend us to believe that this is not defined, 
but our confidence is probably a bit shaken by now. In fact, this multiplication is de- 


fined, and it is 
5 3 -2 
11 5 —4ļ|` 


You may see some similarity in this answer to what we got before, but again, probably 
not enough to really figure things out. 

So let’s take a step back and progress slowly. The first thing we’d like to do is define 
a special type of matrix called a vector. 


Definition 13 Column and Row Vectors 


Am x 1 matrix is called a column vector. 


A 1 x n matrix is called a row vector. 


While it isn’t obvious right now, column vectors are going to become far more use- 
ful to us than row vectors. Therefore, we often omit the word “column” when refering 
to column vectors, and we just call them “vectors.”? 

We have been using upper case letters to denote matrices; we use lower case let- 
ters with an arrow overtop to denote row and column vectors. An example of a row 
vector is 

ü=|1 2 —1 o0] 


2In this text, row vectors are only used in this section when we discuss matrix multiplication, whereas 
we’ll make extensive use of column vectors. Other texts make great use of row vectors, but little use of 
column vectors. It is a matter of preference and tradition: “most” texts use column vectors more. 


2.2 Matrix Multiplication 


and an example of a column vector is 


Before we learn how to multiply matrices in general, we will learn what it means 
to multiply a row vector by a column vector. 


Definition 14 Multiplying a row vector by a column vector 


Let Ube an 1 x n row vector with entries u1, U2, ++, Un and 
let Vbe ann x 1 column vector with entries v1, V2, ***, Vn. 
The product of ui and V, denoted i - V or Uv, is 


n 
y UiVi = U1V1 + UQV2 +--+- + UpnVpy- 
i=l 


Don’t worry if this definition doesn’t make immediate sense. It is really an easy 
concept; an example will make things more clear. 


Example 24 Let 
2 ; 
g@=[1 2 3],v=[2 0 1 -1],¥=] 4], ¥=], 
$ 0 
Find the following products. 
1. ux 3. dy 
5. Xu 
2. Vy 4. uv 
SOLUTION 
—2 
1. ūx=|1 2 3]] 4 | =1(-2)+2(4)4+3(3) =15 
3 
1 
5 2 
2. vy=|2 0 1 -1] z| = 2(1) + 0(2) + 1(5) —1(0) =7 
0 
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3. uy is not defined; Definition 14 specifies that in order to multiply a row vector 
and column vector, they must have the same number of entries. 


4. uv is not defined; we only know how to multipy row vectors by column vec- 
tors. We haven’t defined how to multiply two row vectors (in general, it can’t 
be done). 


5. The product Xu is defined, but we don’t know how to do it yet. Right now, we 
only know how to multiply a row vector times a column vector; we don’t know 
how to multiply a column vector times a row vector. (That’s right: ux Æ xu!) 


Now that we understand how to multiply a row vector by a column vector, we are 
ready to define matrix multiplication. 


Definition 15 Matrix Multiplication 


Let A be an m x r matrix, and let B be an r x n matrix. The 


matrix product of Aand B, denoted A -B, or simply AB, is the 
m x n matrix M whose entry in the i” row and j® column is 
the product of the i” row of A and the j* column of B. 


It may help to illustrate it in this way. Let matrix A have rows dj, d>, ++, Gm and let 
B have columns bı, b2, ---, bp. Thus A looks like 


1 
2 


= Gee, 


where the “—” symbols just serve as reminders that the d; represent rows, and B looks 
like 


| | 
by bs - b, 
| | | 


where again, the “|” symbols just remind us that the b, represent column vectors. Then 


diba diba +-+- dibn 

abı ab, -+> dbn 
AB = 

dmbı dmb2 +> Ambn 


Two quick notes about this definition. First, notice that in order to multiply A and B, 
the number of columns of A must be the same as the number of rows of B (we refer to 
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these as the “inner dimensions”). Secondly, the resulting matrix has the same number 
of rows as A and the same number of columns as B (we refer to these as the “outer 
dimensions”). 


final dimensions are the outer 
dimensions 
(mx r) x (r x n) 
—— 
these inner dimensions 


must match 


Of course, this will make much more sense when we see an example. 


Example 25 Revisit the matrix product we saw at the beginning of this section; 
multiply 
1 2 1 -1 0 
3 4||2 2 —1ļ` 
SOLUTION Let’s call our first matrix A and the second B. We should first check 


to see that we can actually perform this multiplication. Matrix A is 2 x 2 and B is 
2 x 3. The “inner” dimensions match up, so we can compute the product; the “outer” 
dimensions tell us that the product will be 2 x 3. Let 


as = [me M12 ks 


Mı Mzz M23 


Let’s find the value of each of the entries. 
The entry mj, is in the first row and first column; therefore to find its value, we 
need to multiply the first row of A by the first column of B. Thus 


myi=([1 2] B = 1(1) +2(2) =5. 


So now we know that 


5 m m 
AB = 12 13 | 
Mı Mz M233 


Finishing out the first row, we have 


m2=[1 2] [3] = 1(—1) + 2(2) = 3 


using the first row of A and the second column of B, and 


0 


m3=([1 2] [°] = 1(0) + 2(—1) = —2 


using the first row of A and the third column of B. Thus we have 


AB = 5 3 —2 l 
M21 Mz M233 
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To compute the second row of AB, we multiply with the second row of A. We find 


m,=([3 4] H = 11, 


m2=[3 4] BI 25, 


and 
0 
Thus 
1 2 1 -1 0 5 3 -2 
aeh al Taala eal 

Example 26 Multiply 

p F; p 1 1 J 

2 3 2 6 79 

SOLUTION Let’s first check to make sure this product is defined. Again calling 


the first matrix A and the second B, we see that A is a 3 x 2 matrix and Bisa2 x 4 
matrix; the inner dimensions match so the product is defined, and the product will be 
a3 x 4 matrix, 
Mı M2 M130 M14 
AB = | Mmi M2 Mz M24 
M31 M32 M33 M34 


We will demonstrate how to compute some of the entries, then give the final an- 
swer. The reader can fill in the details of how each entry was computed. 


mi=([1 -1]|3]}=-2 
m3=([1 -1]|;]=-6 
m3 = [5 2] | 3] =2. 


m, =[|5 2] B = 23. 
m3 =[-2 3] a = 16. 


ma =[-2 3] B =25. 


2.2 Matrix Multiplication 


So far, we’ve computed this much of AB: 
—-1 mun —-6 my 
AB = | Mi M22 19 23 
M31 16 M33 25 


The final product is 


4 16 19 25 


Example 27 Multiply, if possible, 
2 3 41/3 6 
9 8 7 5 -1]° 
SOLUTION Again, we’ll call the first matrix A and the second B. Checking the 


dimensions of each matrix, we see that A is a 2 x 3 matrix, whereas B is a 2 x 2 matrix. 
The inner dimensions do not match, therefore this multiplication is not defined. 


Example 28 In Example 24, we were told that the product xU was defined, 
where 
—2 
x=| 4 and @=[1 2 3], 
3 


although we were not shown what that product was. Find xu. 


SOLUTION Again, we need to check to make sure the dimensions work cor- 
rectly (remember that even though we are referring to U and x as vectors, they are, in 
fact, just matrices). 

The column vector X has dimensions 3 x 1, whereas the row vector U has dimen- 
sions 1 x 3. Since the inner dimensions do match, the matrix product is defined; the 
outer dimensions tell us that the product will be a 3 x 3 matrix, as shown below: 


Mı Mı «M13 
xu = | Mı Mn M23 
M31 M32 M33 


To compute the entry m11, we multiply the first row of X by the first column of ü. 
What is the first row of X? Simply the number —2. What is the first column of U? Just 
the number 1. Thus mıı = —2. (This does seem odd, but through checking, you can 
see that we are indeed following the rules.) 

What about the entry m12? Again, we multiply the first row of x by the first column 
of u; that is, we multiply —2(2). So my. = —4. 
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What about m23? Multiply the second row of x by the third column of d; multiply 
4(3), so m3 = 12. 

One final example: m3; comes from multiplying the third row of x, which is 3, by 
the first column of ü, which is 1. Therefore m3, = 3. 

So far we have computed 


—2 —4 m3 
xu = Mı Mnr 12 
3 Mzn Mz 


After performing all 9 multiplications, we find 


—2 -4 -6 
xu=}| 4 8 12 
3 6 9 


In this last example, we saw a “nonstandard” multiplication (at least, it felt non- 
standard). Studying the entries of this matrix, it seems that there are several different 
patterns that can be seen amongst the entries. (Remember that mathematicians like 
to look for patterns. Also remember that we often guess wrong at first; don’t be scared 
and try to identify some patterns.) 


In Section 2.1, we identified the zero matrix 0 that had a nice property in relation to 
matrix addition (i.e., A+0 = A for any matrix A). In the following example we'll identify 
a matrix that works well with multiplication as well as some multiplicative properties. 
For instance, we’ve learned how 1- A = A; is there a matrix that acts like the number 
1? That is, can we find a matrix X where X-A = A?? 


Example 29 Let 
1 2 3 1 1 1 
A= 2 -7 5|, B=|1 1 1 
—2 -8 3 111 
1 0 2 1 0 0 
C=|2 1 0|, /=|O 1 0 
O 2 1 O 0 1 
Find the following products. 
1. AB 3. A03x4 5. IA 7. BC 
2. BA 4. Al 6. 1? 8. B? 
SOLUTION We will find each product, but we leave the details of each computa- 


tion to the reader. 


3We made a guess in Section 2.1 that maybe a matrix of all 1s would work. 
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1 2 3 1 1 1 6 6 6 
1,.AB=]} 2 -7 5 1 11;=j] 0 0 0 
—2 -8 3 1 1 1 —7 —7 —7 
1 1 1 1 2 3 1 -13 11 
2. BA= |1 1 1 2 -7 5|=]ļ|1 —13 11 
1 1 1 —2 -8 3 1 -13 11 
3. A03x4 = 03x4. 
1 2 3 1 0 0 1 2 3 
4.Al=} 2 -7 5 0O 1 0;=4]2 =-7 5 
—2 -8 3 001 —2 -8 3 
1 0 0 1 2 3 1 2 «3 
5. A= |O 1 0 2 =7 5; =); 2 -7 5 
001 —2 -8 3 —2 -8 3 


6. We haven't formally defined what I? means, but we could probably make the 
reasonable guess that /? = I - I. Thus 


100 100 100 
P=!01 01/0 1 0] =/]0 10 
o o 1ı|lo 01 001 
111 1 0 2 3 33 3 
7. BC=|1 1 1 21 0/=/3 3 3 
111 0241 3 3 3 
111 111 3 3 3 
8. B =BB=|1 1 1 1 1 1|=1/3 3 3 
1 1 1 1 1 1 3 33 3 


This example is simply chock full of interesting ideas; it is almost hard to think about 
where to start. 

Interesting Idea #1: Notice that in our example, AB +Æ BA! When dealing with 
numbers, we were used to the idea that ab = ba. With matrices, multiplication is 
not commutative. (Of course, we can find special situations where it does work. In 
general, though, it doesn’t.) 

Interesting Idea #2: Right before this example we wondered if there was a matrix 
that “acted like the number 1,” and guessed it may be a matrix of all 1s. However, 
we found out that such a matrix does not work in that way; in our example, AB # A. 
We did find that AJ = JA = A. There is a Multiplicative Identity; it just isn’t what we 
thought it would be. And just as 17 = 1, ? = 1. 

Interesting Idea #3: When dealing with numbers, we are very familiar with the 
notion that “If ax = bx, then a = b.” (As long as x Æ 0.) Notice that, in our example, 
BB = BC, yet B Æ C. In general, just because AX = BX, we cannot conclude that A = B. 
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Matrix multiplication is turning out to be a very strange operation. We are very 
used to multiplying numbers, and we know a bunch of properties that hold when using 
this type of multiplication. When multiplying matrices, though, we probably find our- 
selves asking two questions, “What does work?” and “What doesn’t work?” We’ll an- 
swer these questions; first we’ll do an example that demonstrates some of the things 
that do work. 


Example 30 Let 


Find the following: 


1. A(B+ ©) 3. A(BC) 
2. AB +AC 4. (AB)C 
SOLUTION We’ll compute each of these without showing all the intermediate 


steps. Keep in mind order of operations: things that appear inside of parentheses 
are computed first. 


ae+o=[3 a (a alti al) 
A 4 
[a 20] 
assac= [3 ala alei ai al 
TEDE 
[a 20] 


2.2 Matrix Multiplication 


In looking at our example, we should notice two things. First, it looks like the “dis- 
tributive property” holds; that is, A(B + C) = AB + AC. This is nice as many algebraic 
techniques we have learned about in the past (when doing “ordinary algebra”) will still 
work. Secondly, it looks like the “associative property” holds; that is, A(BC) = (AB)C. 
This is nice, for it tells us that when we are multiplying several matrices together, we 
don’t have to be particularly careful in what order we multiply certain pairs of matrices 
together.’ 

In leading to an important theorem, let’s define a matrix we saw in an earlier ex- 
ample." 


Definition 16 Identity Matrix 


The n x n matrix with 1’s on the diagonal and zeros else- 


where is the n x n identity matrix, denoted /,. When the 
context makes the dimension of the identity clear, the sub- 
script is generally omitted. 


Note that while the zero matrix can come in all different shapes and sizes, the 
identity matrix is always a square matrix. We show a few identity matrices below. 


1 

1 0 
r= a]. s=f0 
O 1 0 


In our examples above, we have seen examples of things that do and do not work. 
We should be careful about what examples prove, though. If someone were to claim 
that AB = BA is always true, one would only need to show them one example where 
they were false, and we would know the person was wrong. However, if someone 


ome) 
a 
l 
ooore 
oroo 
eooo 


ooro 


4Be careful: in computing ABC together, we can first multiply AB or BC, but we cannot change the order 
in which these matrices appear. We cannot multiply BA or AC, for instance. 

5The following definition uses a term we won’t define until Definition 24 on page 137: diagonal. In short, 
a “diagonal matrix” is one in which the only nonzero entries are the “diagonal entries.” The examples given 
here and in the exercises should suffice until we meet the full definition later. 
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claims that A(B + C) = AB + AC is always true, we can’t prove this with just one 
example. We need something more powerful; we need a true proof. 

In this text, we forgo most proofs. The reader should know, though, that when 
we state something in a theorem, there is a proof that backs up what we state. Our 
justification comes from something stronger than just examples. 

Now we give the good news of what does work when dealing with matrix multipli- 
cation. 


Theorem 3 Properties of Matrix Multiplication 


Let A, B and C be matrices with dimensions so that the fol- 
lowing operations make sense, and let k be a scalar. The 
following equalities hold: 


1. A(BC) = (AB)C (Associative Property) 


2. A(B + C) = AB + AB and 
(B + C)A = BA + CA (Distributive Property) 


_ k(AB) = (kA)B = A(kB) 


TA AA 


The above box contains some very good news, and probably some very surprising 
news. Matrix multiplication probably seems to us like a very odd operation, so we 
probably wouldn’t have been surprised if we were told that A(BC) 4 (AB)C. Itis a 
very nice thing that the Associative Property does hold. 

As we near the end of this section, we raise one more issue of notation. We define 
A? = |. If nis a positive integer, we define 


AX =A-A-.-:--A. 
MS 
ntimes 


With numbers, we are used to a™” = = Do negative exponents work with matri- 
ces, too? The answer is yes, sort of. We’ll have to be careful, and we’ll cover the topic 
in detail once we define the inverse of a matrix. For now, though, we recognize the 
fact that A~? Æ + for 1 makes no sense; we don’t know how to “divide” by a matrix. 


We end this section with a reminder of some of the things that do not work with 
matrix multiplication. The good news is that there are really only two things on this 
list. 


1. Matrix multiplication is not commutative; that is, AB Æ BA. 


2. In general, just because AX = BX, we cannot conclude that A = B. 


2.2 Matrix Multiplication 
The bad news is that these ideas pop up in many places where we don’t expect them. 
For instance, we are used to 
(a+b)? =a? + 2ab + b’. 
What about (A + B)?? All we'll say here is that 
(A +B)? + A? + 2AB + B?; 


we leave it to the reader to figure out why. 
The next section is devoted to visualizing column vectors and “seeing” how some 
of these arithmetic properties work together. 


Exercises 2.2 


In Exercises 1 — 12, row and column vectors i 10.¢=[6 2 -1 2] 
and Vare defined. Find the product Uv, where 3 
possible. = 2 
ME 
1.¢=[1 —4] 7=| | 5 
5 4 3 
11 =f 2 <3 = 
2 ī=[2 3] E a= | Jy H 
1 
3. ū=[|1 —1] 7=|3] 12.¢=[2 -5] v=|1 
3 1 
4 a¢=-l06 08 | 0.6 In Exercises 13 — 27, matrices A and B are de- 
= ` ] “= [o8 fined. 
2 (a) Give the dimensions of A and B. If the 
= 5 dimensions properly match, give the 
=|1 2 -1 =|1 
eat á E dimensions of AB and BA. 
(b) Find the products AB and BA, if possi- 
—1 ble. 
6 uU=/]3 2 —2| V= 0 Baza ale 5 
3 NS Ne oa = E 
2 [3 7 i—i 
7.¢=|8 —4 3|vV=ļ|4 14. A = 2 asl, =3 
5 i 
3 —1 
1 15. A= |2 2 | 
8 ū=|-3 6 1|V=|-—1 T1 0 7 
1 B = 
4 2 9 
9% ü=[|1 3 4] 0 1 
1 16. A=| 1 -1 
o fa -2 —4 
v ee p- [-2 ° 
—1 ~ | 3° 8 
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In Exercises 28 — 33, a diagonal matrix D and a 
matrix A are given. Find the products DA and 


AD, where possible. 
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In Exercises 34 — 39, a matrix A and a vector X 0 1 0 f 5 g 
are given. Find the product AX. 43. LetA= |O 0 1). FindA* andA’. 
34. A= cir x= ag ae 
oo fi 1 — J9 
1 4 2 A 
35. A= E s| z= | | 44. LetA= |0 O O|}].FindA*andA’. 
= 0O 1 0 
2 0 3 1 
36. A=|1 1 1), ¥X=ļ4 45. In the text we state that (A + B)? # 
3 —1 2 2 A? +2AB+B?. We investigate that claim 
9 59 8 4 here. 
3. A=| 1 1 -2], *= 1/3 
4 2 -1 1 (a) LetA = E 2 and let B = 
2 1 = xX 
38. A= = = pe 
[a 3 l. as a E 1 57 Compute A + B. 
1 2 3 Xı 
39. A=|1 0 2], K=]x (b) Find (A + B)? by using your an- 
23 1 X3 swer from (a). 
40. Let A = k a Find A? and A’. (c) Compute A? + 2AB + B’. 
2 0 (d) Are the results from (a) and (b) 
41. LetA= f s]: Find A? and A’. the same? 
—1 0 0 (e) Carefully expand the expression 
42. Let A = | 0 3 O}. Find A? and (A+B) = (A+ B)(A +B) 
0 0 5 and show why this is not equal to 
AP. A? + 2AB + B°. 
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2.3 Visualizing Matrix Arithmetic in 2D 


AS YOU READ 


1. T/F: Two vectors with the same length and direction are equal even if they start 
from different places. 


2. One can visualize vector addition using what law? 
3. T/F: Multiplying a vector by 2 doubles its length. 


4. T/F: Multiplying a vector by a matrix always changes its length and direction. 


When we first learned about adding numbers together, it was useful to picture a 
number line: 2 + 3 = 5 could be pictured by starting at 0, going out 2 tick marks, then 
another 3, and then realizing that we moved 5 tick marks from 0. Similar visualizations 
helped us understand what 2 — 3 meant and what 2 x 3 meant. 

We now investigate a way to picture matrix arithmetic — in particular, operations 
involving column vectors. This not only will help us better understand the arithmetic 
operations, it will open the door to a great wealth of interesting study. Visualizing 
matrix arithmetic has a wide variety of applications, the most common being computer 
graphics. While we often think of these graphics in terms of video games, there are 
numerous other important applications. For example, chemists and biologists often 
use computer models to “visualize” complex molecules to “see” how they interact with 
other molecules. 

We will start with vectors in two dimensions (2D) — that is, vectors with only two 
entries. We assume the reader is familiar with the Cartesian plane, that is, plotting 
points and graphing functions on “the x-y plane.” We graph vectors in a manner very 
similar to plotting points. Given the vector 


zel 


we draw xX by drawing an arrow whose tip is 1 unit to the right and 2 units up from its 
er 6 
origin. 


Figure 2.1: Various drawings of x 


To help reduce clutter, in all figures each tick mark represents one unit. 


2.3 Visualizing Matrix Arithmetic in 2D 


When drawing vectors, we do not specify where you start drawing; all we specify 
is where the tip lies based on where we started. Figure 2.1 shows vector X drawn 3 
ways. In some ways, the “most common” way to draw a vector has the arrow start at 
the origin,but this is by no means the only way of drawing the vector. 

Let’s practice this concept by drawing various vectors from given starting points. 


Example 31 Let 


[lB] = [2] 


Draw X starting from the point (0, —1); draw y starting from the point (—1, —1), and 
draw Z starting from the point (2, —1). 


SOLUTION To draw x, start at the point (0, —1) as directed, then move to the 
right one unit and down one unit and draw the tip. Thus the arrow “points” from 
(0, —1) to (1, —2). 

To draw y, we are told to start and the point (—1, —1). We draw the tip by moving 
to the right 2 units and up 3 units; hence y points from (—1, —1) to (1,2). 

To draw Z, we start at (2, —1) and draw the tip 3 units to the left and 2 units up; Z 
points from (2, —1) to (—1, 1). 

Each vector is drawn as shown in Figure 2.2. 


Figure 2.2: Drawing vectors x, y and Z in Example 31 


> 0 ; 
How does one draw the zero vector, 0 = ?7 Following our basic procedure, 


0 
we start by going 0 units in the x direction, followed by O units in the y direction. In 
other words, we don’t go anywhere. In general, we don’t actually draw 0. At best, 
one can draw a dark circle at the origin to convey the idea that 6, when starting at the 
origin, points to the origin. 

In section 2.1 we learned about matrix arithmetic operations: matrix addition and 
scalar multiplication. Let’s investigate how we can “draw” these operations. 


7Vectors are just special types of matrices. The zero vector, 0,isa special type of zero matrix, 0. It helps 
to distinguish the two by using different notation. 
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Vector Addition 


Given two vectors X and y, how do we draw the vector X + y? Let’s look at this in 
the context of an example, then study the result. 


Example 32 Let 
5 1 a |3 
Sketch xX, y and X + y. 
SOLUTION A starting point for drawing each vector was not given; by default, 


we'll start at the origin. (This is in many ways nice; this means that the vector H 


“points” to the point (3,1).) We first compute x + y: 


eer [l+ (a= [2 


Sketching each gives the picture in Figure 2.3. 


Figure 2.3: Adding vectors X and y in Example 32 


This example is pretty basic; we were given two vectors, told to add them together, 
then sketch all three vectors. Our job now is to go back and try to see a relationship 
between the drawings of x, y and x + y. Do you see any? 

Here is one way of interpreting the adding of x to y. Regardless of where we start, 
we draw x. Now, from the tip of x, draw y. The vector x + y is the vector found 
by drawing an arrow from the origin of X to the tip of y. Likewise, we could start by 
drawing y. Then, starting from the tip of y, we can draw x. Finally, draw x + y by 
drawing the vector that starts at the origin of y and ends at the tip of X. 

The picture in Figure 2.4 illustrates this. The gray vectors demonstrate drawing the 
second vector from the tip of the first; we draw the vector x + y dashed to set it apart 
from the rest. We also lightly filled the parallelogram whose opposing sides are the 


vectors X and y. This highlights what is known as the 


xı 


2.3 Visualizing Matrix Arithmetic in 2D 


Parallelogram Law. 


X+y 


Xq 


Figure 2.4: Adding vectors graphically using the Parallelogram Law 


Key Idea 5 Parallelogram Law 


To draw the vector X + y, one 


with X and y as its sides. The 
vertex where x and yý originate 
meet is the vector X + y. 


Knowing all of this allows us to draw the sum o 
specifically what the vectors are, as we demonstrate i 


Example 33 
vector X + y. 


SOLUTION 


xı 
x 


can draw the parallelogram 
vector that points from the 
to the vertex where xX and y 


f two vectors without knowing 
n the following example. 


Consider the vectors X and y as drawn in Figure 2.5. Sketch the 


Figure 2.5: Vectors X and y in Example 33 


We’ll apply the Parallelogram Law, as given in Key Idea 5. As before, we draw X +y 


dashed to set it apart. The result is given in Figure 2.6. 


X+y 


Figure 2.6: Vectors X, yand x + y 


in Example 33 
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Scalar Multiplication 


After learning about matrix addition, we learned about scalar multiplication. We 
apply that concept now to vectors and see how this is represented graphically. 


Example 34 Let 


Sketch x, y, 3X and —1y. 


SOLUTION We begin by computing 3x and —y: 


Ss 3 > (2 
t= (3 and -y7=| 3]; 


All four vectors are sketched in Figure 2.7. 


Figure 2.7: Vectors X, y, 3X and —y in Example 34 


As we often do, let us look at the previous example and see what we can learn from 
it. We can see that x and 3X point in the same direction (they lie on the same line), but 
3x is just longer than X. (In fact, it looks like 3X is 3 times longer than x. Is it? How do 
we measure length?) 

We also see that y and —y seem to have the same length and lie on the same line, 
but point in the opposite direction. 

A vector inherently conveys two pieces of information: length and direction. Mul- 
tiplying a vector by a positive scalar c stretches the vectors by a factor of c; multiplying 
by a negative scalar c both stretches the vector and makes it point in the opposite 
direction. 

Knowing this, we can sketch scalar multiples of vectors without knowing specifi- 
cally what they are, as we do in the following example. 


Example 35 Let vectors X and y be as in Figure 2.8. Draw 3x, —2x, and sy. 
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x 


Xq 


Figure 2.8: Vectors X and y in Example 35 


SOLUTION To draw 3x, we draw a vector in the same direction as X, but 3 times 
as long. To draw —2x, we draw a vector twice as long as X in the opposite direction; 
to draw iy, we draw a vector half the length of y in the same direction as y. We again 
use the default of drawing all the vectors starting at the origin. All of this is shown in 
Figure 2.9. 


< 2X 


Figure 2.9: Vectors X, ¥, 3X, —2x and ix in Example 35 


Vector Subtraction 


The final basic operation to consider between two vectors is that of vector sub- 
traction: given vectors X and y, how do we draw x — y? 

If we know explicitly what x and y are, we can simply compute what x — y is and 
then draw it. We can also think in terms of vector addition and scalar multiplication: 
we can add the vectors x + (—1)y. That is, we can draw x and draw —y, then add them 
as we did in Example 33. This is especially useful we don’t know explicitly what x and 
y are. 


Example 36 Let vectors X and y be as in Figure 2.10. Draw xX — y. 


y 


xı 


Figure 2.10: Vectors X and y in Example 36 


75 


Chapter 2 Matrix Arithmetic 


SOLUTION To draw X — y, we will first draw —y and then apply the Parallelogram 
Law to add x to —y. See Figure 2.11. 


Figure 2.11: Vectors x, y and x — yin Example 36 


In Figure 2.12, we redraw Figure 2.11 from Example 36 but remove the gray vectors 
that tend to add clutter, and we redraw the vector x — y dotted so that it starts from the 
tip of y. Note that the dotted version of X — y points from y to x. This is a “shortcut” 
to drawing X — y; simply draw the vector that starts at the tip of y and ends at the tip 
of X. This is important so we make it a Key Idea. 


y 


Figure 2.12: Redrawing vector X — y 


Key Idea 6 Vector Subtraction 


To draw the vector x — y, draw X and y so that they have the 


same origin. The vector X — y is the vector that starts from 
the tip of y and points to the tip of X. 


Let’s practice this once more with a quick example. 
Example 37 Let X and y be as in Figure 2.13(a). Draw X — y. 


SOLUTION We simply apply Key Idea 6: we draw an arrow from y to x. We do 
so in Figure 2.13; X — y is dashed. 


76 8Remember that we can draw vectors starting from anywhere. 
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<x 
x 


Xq 
X 


(a) (b) 


Figure 2.13: Vectors X, y and x — y in Example 37 


Vector Length 


When we discussed scalar multiplication, we made reference to a fundamental 
question: How do we measure the length of a vector? Basic geometry gives us an 
answer in the two dimensional case that we are dealing with right now, and later we 
can extend these ideas to higher dimensions. 

Consider Figure 2.14. A vector X is drawn in black, and dashed and dotted lines 
have been drawn to make it the hypotenuse of a right triangle. 


Figure 2.14: Measuring the length of a vector 


It is easy to see that the dashed line has length 4 and the dotted line has length 3. 
We'll let c denote the length of x: according to the Pythagorean Theorem, 47+32 = c?. 
Thus c? = 25 and we quickly deduce that c = 5. 

Notice that in our figure, X goes to the right 4 units and then up 3 units. In other 
words, we can write 

> |4 
= B | 


We learned above that the length of x is v42 + 32.° This hints at a basic calculation 
that works for all vectors X, and we define the length of a vector according to this rule. 


Remember that V42 + 32 444 3! 
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Definition 17 Vector Length 


Let 


= X 
x= | “j. 
X2 


The length of x, denoted ||x’||, is 


Ia = a + 


Example 38 Find the length of each of the vectors given below. 
| fee ld + | .6 wen 
ASil ASE ap Sg] MEg 
SOLUTION We apply Definition 17 to each vector. 


Ell = V +12 = v2 
pall = v2? + (-3)? = v13. 
||x3|| = V-62 + .82 = v.36 + .64 = 1. 


Iāll = V32 +0 =3. 


Now that we know how to compute the length of a vector, let’s revisit a statement 
we made as we explored Examples 34 and 35: “Multiplying a vector by a positive scalar 
c stretches the vectors by a factor ofc...” At that time, we did not know how to mea- 
sure the length of a vector, so our statement was unfounded. In the following example, 
we will confirm the truth of our previous statement. 


Example 39 Let X = EN Compute ||x||, ||3x/|, || — 2X1], and ||cx||, where c 
is a scalar. 
SOLUTION We apply Definition 17 to each of the vectors. 
X| = V4 F 1 = V5. 
4) 3 6 
Before computing the length of ||3x/|, we note that 3x = | 3 ; 


||3xX]| = v36 + 9 = v45 = 3V5 = 3||x||. 
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Before computing the length of || — 2x||, we note that —2x = By i 


|| — 2x|| = V16 +4 = v20 = 2v5 =2||x'||. 


‘ > > 2c 
Finally, to compute ||cX||, we note that cx = | | . Thus: 


Ilex] = (20)? + (—c)? = V4e2 + e = v5e = |e|v5. 


This last line is true because the square root of any number squared is the absolute 
value of that number (for example, ,/(—3)? = 3). 


The last computation of our example is the most important one. It shows that, 
in general, multiplying a vector x by a scalar c stretches x by a factor of |c| (and the 
direction will change if c is negative). This is important so we'll make it a Theorem. 


Theorem 4 Vector Length and Scalar Multiplication 


Let X be a vector and let c be a scalar. Then the length of cx 
is 


Ilex] = le] - |x. 


Matrix — Vector Multiplication 


The last arithmetic operation to consider visualizing is matrix multiplication. Specif- 
ically, we want to visualize the result of multiplying a vector by a matrix. In order to 
multiply a 2D vector by a matrix and get a 2D vector back, our matrix must be a square, 
2 x 2 matrix.?° 


We’ll start with an example. Given a matrix A and several vectors, we’ll graph the 
vectors before and after they’ve been multiplied by A and see what we learn. 


Example 40 Let A be a matrix, and x, y, and Z be vectors as given below. 


ES} eG) Ee) LS 


Graph x, y and Z, as well as Ax, Ay and Az. 


SOLUTION 


10We can multiply a 3 x 2 matrix by a 2D vector and get a 3D vector back, and this gives very interesting 
results. See section 5.2. 
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4 Z 
Figure 2.15: Multiplying vectors by a matrix in Example 40. 


It is straightforward to compute: 


3 5 m 3 4 —1 
Aal hs a= [$] and aja 
The vectors are sketched in Figure 2.15 


There are several things to notice. When each vector is multiplied by A, the result 
is a vector with a different length (in this example, always longer), and in two of the 
cases (for y and Z), the resulting vector points in a different direction. 

This isn’t surprising. In the previous section we learned about matrix multiplica- 
tion, which is a strange and seemingly unpredictable operation. Would you expect to 
see some sort of immediately recognizable pattern appear from multiplying a matrix 
and a vector?In fact, the surprising thing from the example is that x and Ax point in 
the same direction! Why does the direction of X not change after multiplication by 
A? (We’ll answer this in Section 4.1 when we learn about something called “eigenvec- 
tors.”) 

Different matrices act on vectors in different ways.‘? Some always increase the 
length of a vector through multiplication, others always decrease the length, others 
increase the length of some vectors and decrease the length of others, and others still 
don’t change the length at all. A similar statement can be made about how matrices 
affect the direction of vectors through multiplication: some change every vector’s di- 
rection, some change “most” vector’s direction but leave some the same, and others 
still don’t change the direction of any vector. 

How do we set about studying how matrix multiplication affects vectors? We could 
just create lots of different matrices and lots of different vectors, multiply, then graph, 
but this would be a lot of work with very little useful result. It would be too hard to 
find a pattern of behavior in this.’ 

Instead, we’ll begin by using a technique we’ve employed often in the past. We 
have a “new” operation; let’s explore how it behaves with “old” operations. Specifi- 
cally, we know how to sketch vector addition. What happens when we throw matrix 


11That’s one reason we call them “different.” 
12Remember, that’s what mathematicians do. We look for patterns. 
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multiplication into the mix? Let’s try an example. 


Example 41 Let A be a matrix and x and y be vectors as given below. 


1 1 = 2 = —1 
aap A e [2] 
Sketch x + y, AX, Ay, and A(x + y). 


SOLUTION It is pretty straightforward to compute: 


z+7=|3|; a= |: a= |9) agy = |3] 


In Figure 2.16, we have graphed the above vectors and have included dashed gray 
vectors to highlight the additive nature of X+ y and A(X + y). Does anything strike you 
as interesting? 


Figure 2.16: Vector addition and matrix multiplication in Example 41. 


Let’s not focus on things which don’t matter right now: let’s not focus on how long 
certain vectors became, nor necessarily how their direction changed. Rather, think 
about how matrix multiplication interacted with the vector addition. 

In some sense, we started with three vectors, X, y, and X + y. This last vector is 
special; it is the sum of the previous two. Now, multiply all three by A. What happens? 
We get three new vectors, but the significant thing is this: the last vector is still the 
sum of the previous two! (We emphasize this by drawing dotted vectors to represent 
part of the Parallelogram Law.) 

Of course, we knew this already: we already knew that Ax + Ay = A(x + y), for 
this is just the Distributive Property. However, now we get to see this graphically. 
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In Section 5.1 we’ll study in greater depth how matrix multiplication affects vec- 
tors and the whole Cartesian plane. For now, we’ll settle for simple practice: given a 
matrix and some vectors, we’ll multiply and graph. Let’s do one more example. 
Example 42 Let A, x, y, and Z be as given below. 

Pe ee zall _ [-1 zala 
Sa saa Aae SEa e a 
Graph x, y and Z, as well as Ax, Ay and AZ. 


SOLUTION 


Ay Y 
Figure 2.17: Multiplying vectors by a matrix in Example 42. 


It is straightforward to compute: 


3 0 m —2 2 3 
“=|. a and z= | 3). 


The vectors are sketched in Figure 2.17. 

These results are interesting. While we won’t explore them in great detail here, 
notice how X got sent to the zero vector. Notice also that Ax, Ay and AZ are all ina 
line (as well as x!). Why is that? Are x, y and Z just special vectors, or would any other 
vector get sent to the same line when multiplied by A??? 


This section has focused on vectors in two dimensions. Later on in this book, we'll 
extend these ideas into three dimensions (3D). 

In the next section we'll take a new idea (matrix multiplication) and apply it to an 
old idea (solving systems of linear equations). This will allow us to view an old idea in 
a new way — and we'll even get to “visualize” it. 


13Don’t just sit there, try it out! 
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2.3 Visualizing Matrix Arithmetic in 2D 


In Exercises 1 — 4, vectors X and y are given. 
Sketch x, y, x+y, and x — y on the same Carte- 


sian axes. 


+-[2)>-[] 


In Exercises 5 — 8, vectors X and y are drawn. 
Sketch 2x, —y, X + y, and X — y on the same 


Cartesian axes. 


y 

X 
5. 

1 
6. 
7. 

x 

8. 1 


y 


In Exercises 9 — 12, a vector X and a scalar 
a are given. Using Definition 17, compute 


the lengths of X and ax, then compare these 
lengths. 


9. X= Ta 
10. X= as 


11. X= |e —1. 


13. Four pairs of vectors X and y are given 
below. For each pair, compute ||X||, 
|ly||, and ||x + y||. Use this information 
to answer: Is it always, sometimes, or 
never true that ||x|| + ||| = |X + y|? 
If it always or never true, explain why. 
If it is sometimes true, explain when it 
is true. 


(a) x= t= 3 


ar-f] [=] 


In Exercises 14 — 17, a matrix A is given. 
Sketch x, y, AX and Ay on the same Cartesian 
axes, where 


re lea and y= Be 


14. A= k | 
15. A = E | 
16. A= i i] 

ae E A 
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2.4 Vector Solutions to Linear Systems 


AS YOU READ 


1. T/F: The equation Ax = b is just another way of writing a system of linear equa- 
tions. 


2. T/F: In solving AX = 0, if there are 3 free variables, then the solution will be 
“pulled apart” into 3 vectors. 


3. T/F: A homogeneous system of linear equations is one in which all of the coeffi- 
cients are 0. 


4. Whether or not the equation Ax = B has a solution depends on an intrinsic 
property of 


The first chapter of this text was spent finding solutions to systems of linear equa- 
tions. We have spent the first two sections of this chapter learning operations that 
can be performed with matrices. One may have wondered “Are the ideas of the first 
chapter related to what we have been doing recently?” The answer is yes, these ideas 
are related. This section begins to show that relationship. 

We have often hearkened back to previous algebra experience to help understand 
matrix algebra concepts. We do that again here. Consider the equation ax = b, where 
a = 3 and b = 6. If we asked one to “solve for x,” what exactly would we be asking? 
We would want to find a number, which we call x, where a times x gives b; in this case, 
it is a number, when multiplied by 3, returns 6. 

Now we consider matrix algebra expressions. We’ll eventually consider solving 
equations like AX = B, where we know what the matrices A and B are and we want to 
find the matrix X. For now, we’ll only consider equations of the type Ax = b, where 
we know the matrix A and the vector b. We will want to find what vector x satisfies 
this equation; we want to “solve for x.” 

To help understand what this is asking, we’ll consider an example. Let 


1 1 1 2 x1 
A=|1 -1 2], b=]-3 and x= | x2 
2 0 1 1 X3 


(We don’t know what x is, so we have to represent it’s entries with the variables x1, x2 
and x3.) Let’s “solve for x,” given the equation Ax = b. 
We can multiply out the left hand side of this equation. We find that 


Xı + X2 + x3 
AX = | x, — X2 + 2x3 


2x1 + X3 


Be sure to note that the product is just a vector; it has just one column. 


2.4 Vector Solutions to Linear Systems 


Since Ax is equal to b, we have 


X1 +X + X3 2 
X1 — X + 2x3 | = | —3 
2x1 + X3 1 


Knowing that two vectors are equal only when their corresponding entries are equal, 
we know 


Xy +X. +X% =2 


Xı — X2 + 2x3 =-3 


2x, + x3 = 1. 


This should look familiar; it is a system of linear equations! Given the matrix-vector 
equation Ax = b, we can recognize A as the coefficient matrix from a linear system 
and b as the vector of the constants from the linear system. To solve a matrix-vector 
equation (and the corresponding linear system), we simply augment the matrix A with 
the vector b, put this matrix into reduced row echelon form, and interpret the results. 

We convert the above linear system into an augmented matrix and find the reduced 
row echelon form: 


1 1 1 2 1 0 O 1 
— 
1 -1 2 -3 ref |O 1 0 2 
2 0 1 1 O0 0O 1 -1 
This tells us that xı = 1, X2 = 2 and x3 = —1, so 
1 
x=|2 
—1 


We should check our work; multiply out AX and verify that we indeed get b: 


1 1 1 1 2 
1 -1 2 2 does equal —3 
2 0 1 —1 1 


We should practice. 


Example 43 Solve the equation Ax = b for X where 
1 2 3 5 
A=j;-1 2 1 and B= |—1 
1 1 0 2 
SOLUTION The solution is rather straightforward, even though we did a lot of 


work before to find the answer. Form the augmented matrix [A b] and interpret its 
reduced row echelon form. 


1 2 3 5 1 0 0 2 
—1 2 1 —1 rref 0100 
110 2 O O 1 1 
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In previous sections we were fine stating that the result as 
x, =2, X2 =0, x3 = 1, 


but we were asked to find x; therefore, we state the solution as 


This probably seems all well and good. While asking one to solve the equation 
Ax = b for X seems like a new problem, in reality it is just asking that we solve a 
system of linear equations. Our variables x1, etc., appear not individually but as the 
entries of our vector x. We are simply writing an old problem in a new way. 

In line with this new way of writing the problem, we have a new way of writing 
the solution. Instead of listing, individually, the values of the unknowns, we simply list 
them as the elements of our vector x. 

These are important ideas, so we state the basic principle once more: solving the 
equation Ax = b for X is the same thing as solving a linear system of equations. Equiv- 
alently, any system of linear equations can be written in the form AX = b for some 
matrix A and vector b. 

Since these ideas are equivalent, we’ll refer to AX = b both as a matrix—vector 
equation and as a system of linear equations: they are the same thing. 

We’ve seen two examples illustrating this idea so far, and in both cases the linear 
system had exactly one solution. We know from Theorem 1 that any linear system has 
either one solution, infinite solutions, or no solution. So how does our new method of 
writing a solution work with infinite solutions and no solutions? 

Certainly, if Ax = B has no solution, we simply say that the linear system has no 
solution. There isn’t anything special to write. So the only other option to consider is 
the case where we have infinite solutions. We’ll learn how to handle these situations 
through examples. 


Example 44 Solve the linear system Ax = 0 for X and write the solution in 
vector form, where 
1 2 > (0) 
A= p A and 0= BE 
SOLUTION (Note: we didn’t really need to specify that 


s[i] 


but we did just to eliminate any uncertainty.) 
To solve this system, put the augmented matrix into reduced row echelon form, 
which we do below. 


2.4 Vector Solutions to Linear Systems 


We interpret the reduced row echelon form of this matrix to write the solution as 


Xı = —2x2 


Xa is free. 


We are not done; we need to write the solution in vector form, for our solution is the 
vector x. Recall that 
= x 
X2 


From above we know that x; = —2x, so we replace the x, in X with —2x>. This gives 


our solution as 
= —2x 
X= sal 
X2 


Now we pull the x2 out of the vector (it is just a scalar) and write X as 


ccs 
xX =X2 1 . 


For reasons that will become more clear later, set 


Thus our solution can be written as 
x= XV. 


Recall that since our system was consistent and had a free variable, we have infinite 
solutions. This form of the solution highlights this fact; pick any value for x) and we 
get a different solution. 

For instance, by setting x2 = —1, 0, and 5, we get the solutions 


es ||P and | 2° 
— |1’ 0? 5 : 
respectively. 


We should check our work; multiply each of the above vectors by A to see if we 
indeed get 0. 

We have officially solved this problem; we have found the solution to AX = O and 
written it properly. One final thing we will do here is graph the solution, using our skills 
learned in the previous section. 

=al] 
xXx=X 1 r 


Our solution is 
: . 5 =2 | . 
This means that any scalar multiply of the vector v = 1 is a solution; we know 


how to sketch the scalar multiples of V. This is done in Figure 2.18. 
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Figure 2.18: The solution, as a line, to AX = 0 in Example 44. 


Here vector V is drawn as well as the line that goes through the origin in the direc- 
tion of V. Any vector along this line is a solution. So in some sense, we can say that the 
solution to AX = Ois a line. 


Let’s practice this again. 


Example 45 Solve the linear system AX = 0 and write the solution in vector 
form, where 
2 -3 
A= . 
SOLUTION Again, to solve this problem, we form the proper augmented ma- 


trix and we put it into reduced row echelon form, which we do below. 


[a Fo] m [o 0" ol 


2 a of T" Jo o ô 
We interpret the reduced row echelon form of this matrix to find that 


Xı = 3/2x2 


Xa is free. 


x= H ; 
X2 
Since x; = 3/2x2, we replace x, in X with 3/2x2: 


7 Ea 


x= 
x2 


As before, 


Now we pull out the x2 and write the solution as 


zaa | 


1 


2.4 Vector Solutions to Linear Systems 


As before, let’s set 


so we can write our solution as 
X = XV. 


Again, we have infinite solutions; any choice of x» gives us one of these solutions. 
For instance, picking x2 = 2 gives the solution 


(This is a particularly nice solution, since there are no fractions. . .) 
As in the previous example, our solutions are multiples of a vector, and hence we 
can graph this, as done in Figure 2.19. 


y 


A 


Y 


Figure 2.19: The solution, as a line, to AX = Õ in Example 45. 


Let’s practice some more; this time, we won’t solve a system of the form AX = 0, 
but instead Ax = b, for some vector b. 


Example 46 Solve the linear system AX = b, where 
1 2 > 3 
a=; A and g= 


SOLUTION (Note that this is the same matrix A that we used in Example 44. 
This will be important later.) 

Our methodology is the same as before; we form the augmented matrix and put it 
into reduced row echelon form. 


£23) oe [hoa 
246| "™ Jooo 
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Interpreting this reduced row echelon form, we find that 


X, = 3- 2x) 


Xa is free. 


= Xı 
x= 5 
X2 


and we replace x; with 3 — 2x), giving 


z- Po 
X2 


This solution is different than what we’ve seen in the past two examples; we can’t 
simply pull out a x2 since there is a 3 in the first entry. Using the properties of matrix 
addition, we can “pull apart” this vector and write it as the sum of two vectors: one 
which contains only constants, and one that contains only “xz stuff.” We do this below. 


a 
Jl 
_ 0 e Fak 


Once again, let’s give names to the different component vectors of this solution 
(we are getting near the explanation of why we are doing this). Let 


s[e] = [2] 


We can then write our solution in the form 


Again, 


X = Xp + XV. 


We still have infinite solutions; by picking a value for x2 we get one of these solu- 
tions. For instance, by letting x2 = —1, 0, or 2, we get the solutions 


A} [eo] a [E] 


We have officially solved the problem; we have solved the equation AX = b for x 
and have written the solution in vector form. As an additional visual aid, we will graph 
this solution. 

Each vector in the solution can be written as the sum of two vectors: x, and a 
multiple of V. In Figure 2.20, xp is graphed and Vis graphed with its origin starting at 
the tip of xp. Finally, a line is drawn in the direction of Vv from the tip of xp; any vector 
pointing to any point on this line is a solution to Ax = b. 


2.4 Vector Solutions to Linear Systems 


i 
Figure 2.20: The solution, as a line, to AX = b in Example 46. 


The previous examples illustrate some important concepts. One is that we can 
“see” the solution to a system of linear equations in a new way. Before, when we had 
infinite solutions, we knew we could arbitrarily pick values for our free variables and 
get different solutions. We knew this to be true, and we even practiced it, but the 
result was not very “tangible.” Now, we can view our solution as a vector; by picking 
different values for our free variables, we see this as multiplying certain important 
vectors by a scalar which gives a different solution. 

Another important concept that these examples demonstrate comes from the fact 
that Examples 44 and 46 were only “slightly different” and hence had only “slightly 
different” answers. Both solutions had 


in them; in Example 46 the solution also had another vector added to this. Was this 
coincidence, or is there a definite pattern here? 
Of course there is a pattern! Now ... what exactly is it? First, we define a term. 


Definition 18 Homogeneous Linear System of Equations 


A system of linear equations is homogeneous if the con- 
stants in each equation are zero. 


Note: a homogeneous system of equations can be written 
in vector form as AX = 0. 


The term homogeneous comes from two Greek words; homo meaning “same” and 
genus meaning “type.” A homogeneous system of equations is a system in which each 
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equation is of the same type — all constants are 0. Notice that the system of equations 
in Examples 44 and 46 are homogeneous. 

Note that AO = 0; that is, if we set X = 0, we have a solution to a homogeneous set 
of equations. This fact is important; the zero vector is always a solution to a homoge- 
neous linear system. Therefore a homogeneous system is always consistent; we need 
only to determine whether we have exactly one solution (just 0) or infinite solutions. 
This idea is important so we give it it’s own box. 


Key Idea 7 Homogeneous Systems and Consistency 


All homogeneous linear systems are consistent. 


How do we determine if we have exactly one or infinite solutions? Recall Key Idea 
2: if the solution has any free variables, then it will have infinite solutions. How can 
we tell if the system has free variables? Form the augmented matrix [A O], put it 
into reduced row echelon form, and interpret the result. 

It may seem that we’ve brought up a new question, “When does AX = O have ex- 
actly one or infinite solutions?” only to answer with “Look at the reduced row echelon 
form of A and interpret the results, just as always.” Why bring up a new question if the 
answer is an old one? 

While the new question has an old solution, it does lead to a great idea. Let’s 
refresh our memory; earlier we solved two linear systems, 


AX=0 and Ax=b 


1 2 > 3 
iaj, a and g=]: 


The solution to the first system of equations, AX = 0, is 


Pel 
X =X? 1 


and the solution to the second set of equations, Ax = b, is 


ed +x ae 
~ 10 es. |? 
for all values of x2. 


Recalling our notation used earlier, set 


3 3 X —2 
X si | and let E |. 


where 


2.4 Vector Solutions to Linear Systems 


Thus our solution to the linear system AX = bis 


Let us see how exactly this solution works; let’s see why Ax equals b. Multiply AX: 


AX = A(Xp + X2V) 
= Ax, + A(x2V) 
= AX, + X2(AV) 


= Ax, + x0 
= AX, + 0 
= AX% 

=b 


We know that the last line is true, that Ax, = b, since we know that x was a solution 
to AX = b. The whole point is that x; itself is a solution to Ax = b, and we could find 
more solutions by adding vectors “that go to zero” when multiplied by A. (The subscript 
p of “x,” is used to denote that this vector is a “particular” solution.) 


Stated in a different way, let’s say that we know two things: that Ax, = b and 
AV = 0. What is A(x, + V)? We can multiply it out: 


A(Xp + V) = Ax, + AV 


b+0 


II 
© 


and see that A(x, + V) also equals b. 


So we wonder: does this mean that AX = b will have infinite solutions? After all, 
if x, and x, + V are both solutions, don’t we have infinite solutions? 


No. If AX = O has exactly one solution, then Vv = 0, and x, = x, + V; we only have 
one solution. 


So here is the culmination of all of our fun that started a few pages back. If V is 
a solution to AX = O and x; is a solution to AX = b, then x, + 7 is also a solution to 
AX = b. If AX = O has infinite solutions, so does Ax = b: if Ax = O has only one 
solution, so does Ax = B. This culminating idea is of course important enough to be 
stated again. 


93 


Chapter 2 Matrix Arithmetic 


Key Idea 8 Solutions of Consistent Systems 


Let AX = b be a consistent system of linear equations. 


1. If Ax = 0 has exactly one solution (x = 0), then Ax = 
b has exactly one solution. 


2. If Ax = Ohas infinite solutions, then Ax = b has infi- 
nite solutions. 


A key word in the above statement is consistent. If AX = b is inconsistent (the 
linear system has no solution), then it doesn’t matter how many solutions AX = 0 has; 
Ax = b has no solution. 

Enough fun, enough theory. We need to practice. 


Example 47 Let 


Solve the linear systems Ax = O and Ax = b for X, and write the solutions in vector 
form. 


SOLUTION We'll tackle Ax = O first. We form the associated augmented ma- 
trix, put it into reduced row echelon form, and interpret the result. 


1-113 0) zœ |101 2 0 
4 2 4 6 0 010 -1 0 


Xı = —X3 — 2X4 
X2 = X4 
X3 is free 


X4 is free 


To write our solution in vector form, we rewrite x, and x in X in terms of x3 and x4. 


X1 —X3 — 2X4 
> X2 X4 
x= = 

X3 X3 

X4 X4 


Finally, we “pull apart” this vector into two vectors, one with the “x3 stuff” and one 
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with the “x, stuff.” 


—X3 — 2X4 
x= a 
X3 
X4 
—X3 —2X4 
0 X4 
E X3 + 0 
0 X4 
—1 —2 
0 4 1 
5B] Mi |T] o 
0 1 
= xX3U + X4V 


We use U and V simply to give these vectors names (and save some space). 


It is easy to confirm that both U and V are solutions to the linear system AX = 0. 
(Just multiply Au and Av and see that both are 0.) Since both are solutions to a ho- 
mogeneous system of linear equations, any linear combination of @ and V will be a 
solution, too. 


Now let’s tackle Ax = b. Once again we put the associated augmented matrix into 
reduced row echelon form and interpret the results. 


i 43 Gf) oe fia 2 2 
4 2 4.6.40) 7" Jo 1 o 1 1 


Xı = 2 — x3 — 2X4 
Xx =1+x, 
X3 is free 


X4 is free 


Writing this solution in vector form gives 


X1 2 — X3 — 2X4 
= X 1+x 
x= 2 = 4 

X3 X3 

X4 X4 


Again, we pull apart this vector, but this time we break it into three vectors: one with 
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a“ 


X3” stuff, one with “x,” stuff, and one with just constants. 


2 — x3 — 2X4 


ž — 1 + X4 
X3 
X4 
2 —X3 —2x4 
1 0 X4 
(0) T X3 + 0 
0 0 X4 
2 —1 —2 
1 0 1 
Slol ae t] o 
0 0 1 


= p + X3U + X4V 

“HY = 
particular solution to 

solution homogeneous 


equations Ax = 0 


Note that Ax, = b; by itself, Xp is a solution. To get infinite solutions, we add a bunch 
of stuff that “goes to zero” when we multiply by A; we add the solution to the homo- 
geneous equations. 


Why don’t we graph this solution as we did in the past? Before we had only two 
variables, meaning the solution could be graphed in 2D. Here we have four variables, 
meaning that our solution “lives” in 4D. You can draw this on paper, but it is very con- 
fusing. 


Example 48 Rewrite the linear system 
Xx + 2x2 — 3x3 + 2x4 + 7% = 2 
3x, + 4x. + 5x3 + 2x4 + 3x5 = —4 


as a matrix-vector equation, solve the system using vector notation, and give the so- 
lution to the related homogeneous equations. 


SOLUTION Rewriting the linear system in the form of AX = b, we have that 


| 


1 2 -3 2 7 = > 2 
A= 34 5 2 ii X= | X3 and b= (2, 
X4 
X5 


To solve the system, we put the associated augmented matrix into reduced row eche- 
lon form and interpret the results. 


2.4 Vector Solutions to Linear Systems 


1 2 -3 2 7 A — HETET 
0O 1 -7 2 9 5 

Xı = —8 — 11x; + 2x, + 11x5 

X2 = 5 + 7x3 — 2x4 — 9X5 

X3 is free 

X4 is free 

Xs is free 


We use this information to write x, again pulling it apart. Since we have three free 
variables and also constants, we’ll need to pull X apart into four separate vectors. 


xı 
II 
x 

w 


—8 — 11x3 + 2x, + 11x5 
5 + 7x3 — 2x4 — 9X5 


= X3 
X4 
X5 
—8 —11x3 2x4 11x5 
5 7X3 —2x4 —9xs 
=| Ola) x |+| 0 |+| 0 
0 0 Xa 0 
0 0 0 X5 
—8 —11 2 11 
=] 0 | +x3} 1 | +x] 0 | +x5] 0 
0 0 1 (0) 
0 (0) 0 1 
= xX + X3U + X4V + x5W 
“J =-rev--_-—’ 
particular solution to homogeneous 
solution equations Ax = 0 


So Xp is a particular solution; Ax, = b. (Multiply it out to verify that this is true.) 
The other vectors, U, V and w, that are multiplied by our free variables x3, xq and xs, 
are each solutions to the homogeneous equations, Ax = 0. Any linear combination 
of these three vectors, i.e., any vector found by choosing values for x3, X4 and xs in 
x3U + x,V + xs W is a solution to Ax = 0. 
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Example 49 Let 


Find the solutions to Ax = b and Ax = O. 


SOLUTION We go through the familiar work of finding the reduced row eche- 
lon form of the appropriate augmented matrix and interpreting the solution. 


1 2 3) Fe |10 -1 
4 5 6 O 1 2 


Thus 


This may strike us as a bit odd; we are used to having lots of different vectors in the 
solution. However, in this case, the linear system Ax = b has exactly one solution, and 
we’ve found it. What is the solution to AX = 0? Since we’ve only found one solution to 
AX = b, we can conclude from Key Idea 8 the related homogeneous equations Ax = 0 
have only one solution, namely x = 0. We can write our solution vector X in a form 
similar to our previous examples to highlight this: 


i =F] a] 9 
2 0 

= % + 6 
“SY “SY 
particular solution to 
solution Ax =0 


Example 50 Let 
1 1 > 1 
A= p J and b= HE 
Find the solutions to Ax = b and Ax = O. 


SOLUTION To solve AX = b, we put the appropriate augmented matrix into 
reduced row echelon form and interpret the results. 


tO ale [i PO 
22 al! "F Joo a 


98 


2.4 Vector Solutions to Linear Systems 


We immediately have a problem; we see that the second row tells us that Ox; + 
Ox, = 1, the sign that our system does not have a solution. Thus Ax = B has no 
solution. Of course, this does not mean that AX = 0 has no solution; it always has a 
solution. 

To find the solution to Ax = 0, we interpret the reduced row echelon form of the 
appropriate augmented matrix. 

1 1 0 — 1 1 0 
p 2 4 di E 0 | 


Xı = —X2 


X is free 


Thus 


= Xü. 
We have no solution to Ax = b, but infinite solutions to AX = 0. 


The previous example may seem to violate the principle of Key Idea 8. After all, 
it seems that having infinite solutions to AX = 0 should imply infinite solutions to 
AX =b. However, we remind ourselves of the key word in the idea that we observed 
before: consistent. If AX = b is consistent and AX = G has infinite solutions, then so 
will Ax = b. But if Ax = b is not consistent, it does not matter how many solutions 
AX = Ohas; AX = bis still inconsistent. 


This whole section is highlighting a very important concept that we won’t fully un- 
derstand until after two sections, but we get a glimpse of it here. When solving any 
system of linear equations (which we can write as AX = b), whether we have exactly 
one solution, infinite solutions, or no solution depends on an intrinsic property of A. 
We'll find out what that property is soon; in the next section we solve a problem we 
introduced at the beginning of this section, how to solve matrix equations AX = B. 


Exercises 2.4 


In Exercises 1 — 6, a matrix A and vectors b, u 1. A= | 1 ral 

and V are given. Verify that U and V are both -3 6 |’ 

solutions to the equation AX = b: that is, E ol . 2] —10 
show that AŬ = Av = b. eje T 
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1 —2 —1 2 2 
AT Pa a b= -|a= 2|,v=]|3 
sale odie ee: 1 0 0 
~{=—6]'°  |—11’" |o 20 0 
9 A=]|0 1 -3 
1 0 v 
3. A= 2 a 3 1 -3 
p= oj- _ 10] _ =| 0 3 2 g 0 a 1 
=| ee = | aa Y=] s9 b=|-4],u=]6],v=]|-1 
—1 2 1 
1 0 5 
4. A= > ol In Exercises 10 — 24, a matrix A and vector b 
are given. 
a 37) 3-0 RS) Se: 3 f Pees 
=) 6) 4=]_4}-%=] 59 (a) Solve the equation Ax = 0. 
(b) Solve the equation Ax = b. 


11 
~ fol. 4 
b= o i= -a2 l 
0 
9 
ou, | —2 
=| o 
12 
Oe SF: i 3 
aod et tae, ae ee 
=17 
-> [48] _ —16 
25 ee OF) 
0 
-8 
z- | -28 
~1 9 
12 


In Exercises 7 —9, a matrix A and vectors b, 
and V are given. Verify that AU = 0, AV = 


and A(ū + 7) = b. 
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In each of the above, be sure to write your an- 
swer in vector format. Also, when possible, 
give 2 particular solutions to each equation. 


10. a=] ° oE 
11. a= |72 ef 


manfa z-e] 
| 


13. A= 


[2 -3] > 
a= (2, 2) a=[4] 


1s. a=| 74 : 3 -8= [oa 
6. =|} 7 yeaa 
va=|F 5 ahei] 
saie 3 ae] 
1. a= li 3 s ar 


-4 2 -5 4 

ee ANG 1 -1 af 
+ [-3 
| 


2.5 Solving Matrix Equations AX = B 


21. A= 


In Exercises 25 — 28, a matrix A and vector b 
are given. Solve the equation Ax = b, write 
the solution in vector format, and sketch the 
solution as the appropriate line on the Carte- 


| 
| 
: 
7 z san plane 
| 
| 
| 


22. A= 


4 f2 4]. fo 
25. A= b= 
-1 3 1 -3 4 |—1 = al 
23. A=| 3 -3 -1 1 4), E 
> 3 —2 —3 1 -|2 Aol os a6 
i 26. A= | A 3 


24. A= 


2.5 Solving Matrix Equations AX = B 


AS YOU READ 


1. T/F: To solve the matrix equation AX = B, put the matrix [A X] into reduced 
row echelon form and interpret the result properly. 


2. T/F: The first column of a matrix product AB is A times the first column of B. 


3. Give two reasons why one might solve for the columns of X in the equation AX=B 
separately. 


We began last section talking about solving numerical equations like ax = b for x. 
We mentioned that solving matrix equations of the form AX = B is of interest, but we 
first learned how to solve the related, but simpler, equations Ax = b. In this section 
we will learn how to solve the general matrix equation AX = B for X. 

We will start by considering the best case scenario when solving Ax = b; that is, 
when A is square and we have exactly one solution. For instance, suppose we want to 


solve Ax = b where 
1 1 > (0) 
a=; | and p=]: 


101 


Chapter 2 Matrix Arithmetic 


102 


We know how to solve this; put the appropriate matrix into reduced row echelon form 
and interpret the result. 


We read from this that 


Written in a more general form, we found our solution by forming the augmented 

matrix 
[A b] 
and interpreting its reduced row echelon form: 
3 — x% 
[A b] rref [I x] 

Notice that when the reduced row echelon form of A is the identity matrix / we have 
exactly one solution. This, again, is the best case scenario. 

We apply the same general technique to solving the matrix equation AX = B for X. 
We’ll assume that A is a square matrix (B need not be) and we’ll form the augmented 


matrix 
[A B]. 


Putting this matrix into reduced row echelon form will give us X, much like we found x 
before. 

— 
rref 


[a 8] mè [1 x] 


As long as the reduced row echelon form of A is the identity matrix, this technique 
works great. After a few examples, we’ll discuss why this technique works, and we’ll 
also talk just a little bit about what happens when the reduced row echelon form of A 
is not the identity matrix. 

First, some examples. 


Example 51 Solve the matrix equation AX = B where 
1 —1 —8 -13 1 
a E 3 | ae S E -17 ak 
SOLUTION To solve AX = B for X, we form the proper augmented matrix, put 


it into reduced row echelon form, and interpret the result. 


1-1 -8 -13 1/ z |10 1 -7 3 
5 3 32 —17 21 019 6 2 


We read from the reduced row echelon form of the matrix that 


1 -7 3 
x=; 6 F 


2.5 Solving Matrix Equations AX = B 


We can easily check to see if our answer is correct by multiplying AX. 


Example 52 Solve the matrix equation AX = B where 
1 0 2 —1 2 
A=|0 -1 -2 and B=| 2 —6 
2 —1 0 2 —4 
SOLUTION To solve, let’s again form the augmented matrix 
[A B], 


put it into reduced row echelon form, and interpret the result. 


1 0 2 —1 2 100 1 0 
0o -1 -2 2 -6| mẹ lo0o10 0 4 
2 -1 0 2 —4 0 0 1 —1 1 


We see from this that 


Why does this work? To see the answer, let’s define five matrices. 


1 2] ,_ fa] & [-1] _ f5 ofi -1 5 
a a [a] a ex=[2 a | 


Notice that ü, V and W are the first, second and third columns of X, respectively. 
Now consider this list of matrix products: Au, AV, Aw and AX. 


So again note that the columns of X are u, V and w; that is, we can write 


x=|ū v w]. 103 
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Notice also that the columns of AX are Au, AV and Aw, respectively. Thus we can write 


We summarize what we saw above in the following statement: 
The columns of a matrix product AX are A times the columns of X. 


How does this help us solve the matrix equation AX = B for X? Assume that Ais a 
square matrix (that forces X and B to be the same size). We’ I let Xi, X2,- Xn denote 
the columns of the (unknown) matrix X, and we'll let bi, b3,- -b;, denote the columns 
of B. We want to solve AX = B for X. That is, we want X where 


AX =B 
aji % a]=[b b = bn] 
[Axi Ax Ax, | =[b1 b2 -> bn] 


If the matrix on the left hand side is equal to the matrix on the right, then their 
respective columns must be equal. This means we need to solve n equations: 


Axi = by 
AX = b> 
AX, = n 


We already know how to do this; this is what we learned in the previous section. 
Let’s do this in a concrete example. In our above work we defined matrices A and X, 
and looked at the product AX. Let’s call the product B; that is, set B= AX. Now, let’s 
pretend that we don’t know what X is, and let’s try to find the matrix X that satisfies 
the equation AX = B. As a refresher, recall that 


T 2 
Since A is a2 x 2 matrix and B is a2 x 3 matrix, what dimensions must X be in the 
equation AX = B? The number of rows of X must match the number of columns of A; 


the number of columns of X must match the number of columns of B. Therefore we 
know that X must be a2 x 3 matrix. 


2.5 Solving Matrix Equations AX = B 


We'll call the three columns of X x7, X and x3. Our previous explanation tells us 
that if AX = B, then: 


AX = B 
ali a le(is] 
[Axi AX al =| i A 


Hence 


To find x;, we form the proper augmented matrix and put it into reduced row ech- 
elon form and interpret the results. 


123)" a> ila E 
3 4 7| " Jo 


aeli 


To find x2, we again form an augmented matrix and interpret its reduced row ech- 


elon form. 
1 2 1 Ger 1 0 -1 
3 41| EES Jo1 1 


gala 
2 |1 
which matches with what we already knew from above. 


Before continuing on in this manner to find x3, we should stop and think. If the 
matrix vector equation Ax = b is consistent, then the steps involved in putting 


This shows us that 


Thus 


[A b] 


into reduced row echelon form depend only on A; it does not matter what bis. So 
when we put the two matrices 


12 3 rhai 
a g g an 341 
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from above into reduced row echelon form, we performed exactly the same steps! (In 
fact, those steps are: —3R, + Ry > Ro; —}R2 — Rz; —2R3 + Ri > Rr.) 

Instead of solving for each column of X separately, performing the same steps to 
put the necessary matrices into reduced row echelon form three different times, why 


don’t we just do it all at once? Instead of individually putting 
1 2 3 1 2 1 d 1 2 17 
3 4 7’ 3 4 1 3 4 39 


into reduced row echelon form, let’s just put 


t 2 3 1 17 
3 4 7 1 39 


into reduced row echelon form. 


1 2 3 1 17| —> |1 0 1 —1 5 
3 4 7 1 39 


By looking at the last three columns, we see X: 
1 -1 5 
x-i T el 
Now that we’ve justified the technique we’ve been using in this section to solve 
AX = B for X, we reinfornce its importance by restating it as a Key Idea. 
Key Idea 9 Solving AX = B 


Let A be an n x n matrix, where the reduced row echelon 
form of A is I. To solve the matrix equation AX = B for X, 


1. Form the augmented matrix [A B]. 


2. Put this matrix into reduced row echelon form. It 
will be of the form [/ X], where X appears in the 
columns where B once was. 


These simple steps cause us to ask certain questions. First, we specify above that A 
should be a square matrix. What happens if A isn’t square? Is a solution still possible? 
Secondly, we only considered cases where the reduced row echelon form of A was / 
(and stated that as a requirement in our Key Idea). What if the reduced row echelon 
form of A isn’t I? Would we still be able to find a solution? (Instead of having exactly 
one solution, could we have no solution? Infinite solutions? How would we be able to 
tell?) 

These questions are good to ask, and we leave it to the reader to discover their 
answers. Instead of tackling these questions, we instead tackle the problem of “Why 


2.6 The Matrix Inverse 


do we care about solving AX = B?” The simple answer is that, for now, we only care 
about the special case when B = I. By solving AX = I for X, we find a matrix X that, 
when multiplied by A, gives the identity /. That will be very useful. 


Exercises 2.5 


In Exercises 1 — 12, matrices A and B are B= E —10 19 | 
given. Solve the matrix equation AX = B. 3 2 —2 
4 -1 1 O 
rasli q: za i B=h 
8 —31 
B= | | 2 2 
—27 38 .A= = 
8. A E a B=! 
1 -3 
2. A= E 6 l —2 0 4 
49 40 9A=]-5 -4 5 |], 
B= p -3 5 —3 
—27 27 
—18 2 —14 
3. A= 3 3 l B= | —38 18 —13 
6 4 10 2 —18 
B= 15 —39 
= |16 —66 oS a 
10. A= | 8 —2 -3], 
— — 6 1 -8 
THe 
0 —21 -8 -19 
B= 48 —30 B= | 65 —11 —10 
~ 10 -8 75 —51 33 
T e ah 0 -2 1 
—2 —-3 11,.A=]0 2 2 |, B=kh 
B= 13 4 7 1 2 —3 
~ 122 5 12 
-3 3 -2 
_{|-4 1 12. A = 1 -3 2 |, B=h 
sa|- al —1 -1 2 


2.6 The Matrix Inverse 


AS YOU READ 


1. T/F: If A and B are square matrices where AB = I, then BA = 1. 
2. T/F: A matrix A has exactly one inverse, infinitely many inverses, or no inverse. 
3. T/F: If A is invertible, then AX = Ohas exactly 1 solution. 


4. What is a corollary? 
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5. Fill in the blanks: a matrix is invertible is useful; computing the inverse 
is 


Once again we visit the old algebra equation, ax = b. How do we solve for x? We 
know that, as long as a Æ 0, 


b 
x = —, or, stated in another way, x = a+b. 
a 


What is a71? It is the number that, when multiplied by a, returns 1. That is, 
a -a=1. 


Let us now think in terms of matrices. We have learned of the identity matrix / that 
“acts like the number 1.” That is, if A is a square matrix, then 


IA=AI=A. 


If we hada matrix, which we'll callA~*, where ATTA = 1, then by analogy to our algebra 
example above it seems like we might be able to solve the linear system Ax = b for X 
by multiplying both sides of the equation by A~?. That is, perhaps 


Xx=A~'b. 


Of course, there is a lot of speculation here. We don’t know that such a matrix like 
A~' exists. However, we do know how to solve the matrix equation AX = B, so we 
can use that technique to solve the equation AX = I for X. This seems like it will get us 
close to what we want. Let’s practice this once and then study our results. 


Example 53 Let 
2 1 
A — 
Find a matrix X such that AX = /. 
SOLUTION We know how to solve this from the previous section: we form 


the proper augmented matrix, put it into reduced row echelon form and interpret the 


results. 
2 1 1 0 ret 1 0 1 —1 
1 1 O 1 Oo 1 -1 2 


We read from our matrix that 
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Let’s check our work: 


Sure enough, it works. 


Looking at our previous example, we are tempted to jump in and call the matrix X 
that we found “AT!” However, there are two obstacles in the way of us doing this. 

First, we know that in general AB Æ BA. So while we found that AX = I, we can’t 
automatically assume that XA = /. 

Secondly, we have seen examples of matrices where AB = AC, but B + C. So if we 
find matrices X and Y such that AX = I and AY = I, we cannot automatically conclude 
that X = Y (yet). If this is the case, using the notation A~*? would be misleading, since 
it could refer to more than one matrix. 

These obstacles that we face are not insurmountable. The first obstacle was that 
we know that AX = / but didn’t know that XA = I. That’s easy enough to check, 
though. Let’s look at A and X from our previous example. 


m=[ 2 FYE al 


Perhaps this first obstacle isn’t much of an obstacle after all. Of course, we only 
have one example where it worked, so this doesn’t mean that it always works. We 
have good news, though: it always does work. The only “bad” news to come with this 
is that this is a bit harder to prove. We won’t worry about proving it always works, but 
state formally that it does in the following theorem. 


Theorem 5 Special Commuting Matrix Products 


Let A be ann x n matrix. 


1. If there is a matrix X such that AX = hn, then XA = Ip. 


2. If there is a matrix X such that XA = ln, then AX = hn. 


The second obstacle is easier to address. We want to know if another matrix Y 


exists where AY = | = YA. Let’s suppose that it does. Consider the expression XAY. 
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Since matrix multiplication is associative, we can group this any way we choose. We 
could group this as (XA)Y; this results in 


We could also group XAY as X(AY). This tells us 


X(AY) = XI 
=X 


Combining the two ideas above, we see that X = XAY = Y; that is, X = Y. We 
conclude that there is only one matrix X where XA = I = AX. (Even if we think we 
have two, we can do the above exercise and see that we really just have one.) 

We have just proved the following theorem. 


Theorem 6 Uniqueness of Solutions to AX = ln 


Let A be ann x n matrix and let X be a matrix where AX = 
ln. Then X is unique; it is the only matrix that satisfies this 
equation. 


So given a square matrix A, if we can find a matrix X where AX = /, then we know 
that XA = / and that X is the only matrix that does this. This makes X special, so we 
give it a special name. 


Definition 19 Invertible Matrices and the Inverse of A 


Let A and X be n x n matrices where AX = | = XA. Then: 


1. Ais invertible. 


2. Xis the inverse of A, denoted by A~?. 


Let’s do an example. 


1 2 
Example 54 Find the inverse of A = p A 
SOLUTION By solving the equation AX = I for X will give us the inverse of A. 


Forming the appropriate augmented matrix and finding its reduced row echelon form 


110 


2.6 The Matrix Inverse 


gives us 

1 2 10 —. 1 2 0 1/2 

rref 

24 0 1 001 —1/2 
Yikes! We were expecting to find that the reduced row echelon form of this matrix 
would look like 

[1 A]. 

However, we don’t have the identity on the left hand side. Our conclusion: A is not 
invertible. 


We have just seen that not all matrices are invertible.With this thought in mind, 
let’s complete the array of boxes we started before the example. We’ve discovered 
that if a matrix has an inverse, it has only one. Therefore, we gave that special matrix 
a name, “the inverse.” Finally, we describe the most general way to find the inverse of 
a matrix, and a way to tell if it does not have one. 


Key Idea 10 Finding A~‘! 


Let A be ann x n matrix. To find A~?, put the augmented 
matrix 


[a n] 


into reduced row echelon form. If the result is of the form 


[h x], 


then AT! = X. If not, (that is, if the first n columns of the re- 
duced row echelon form are not /,), then A is not invertible. 


Let’s try again. 


1 1 —1 
Example 55 Find the inverse, if it exists, f A= |1 —1 1 
1 2 3 
SOLUTION We'll try to solve AX = I for X and see what happens. 
1 1 —1 1 0 0 ae 100 05 0.5 0 
1-1 1 0 1 0 rref 010 02 —0.4 0.2 
1 2 3 001 001 -03 0.1 0.2 


0.5 0.5 0 
A=j| 0.2 —0.4 0.2 
—0.3 01 0.2 
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Multiply AAT! to verify that it is indeed the inverse of A. 

In general, given a matrix A, to find A~* we need to form the augmented matrix 
[A 1] and put it into reduced row echelon form and interpret the result. In the case 
of a 2 x 2 matrix, though, there is a shortcut. We give the shortcut in terms of a 
theorem.“ 


Theorem 7 The Inverse of a 2x2 Matrix 


Let 


A is invertible if and only if ad — bc Æ 0. 


If ad — bc Æ 0, then 


We can’t divide by 0, so if ad — bc = 0, we don’t have an inverse. Recall Example 


54, where 
1 2 
ia 


Here, ad — bc = 1(4) — 2(2) = 0, which is why A didn’t have an inverse. 
Although this idea is simple, we should practice it. 


Example 56 Use Theorem 7 to find the inverse of 
3 2 
a=, a 
if it exists. 
SOLUTION Since ad — bc = 29 Æ 0, A~' exists. By the Theorem, 
Ata 1 9 -2 
3(9) —2(-1) |1 3 
T i 9 -2 
~ 29{1 3 


14We don’t prove this theorem here, but it really isn’t hard to do. Put the matrix 
a b 1 0 
c d 0 1 


into reduced row echelon form and you’ll discover the result of the theorem. Alternatively, multiply A by 
112 what we propose is the inverse and see that we indeed get /. 


2.6 The Matrix Inverse 


We can leave our answer in this form, or we could “simplify” it as 


ee eee aera 


We started this section out by speculating that just as we solved algebraic equa- 
tions of the form ax = b by computing x = a~+b, we might be able to solve matrix 
equations of the form Ax = b by computing x = A~1p. If A7! does exist, then we can 
solve the equation Ax = b this way. Consider: 


(original equation) 


b 
E = Atb (multiply both sides on the left by A™*) 

Atb (since A™*A = 1) 

Ath (since IX = X) 


Let’s step back and think about this for a moment. The only thing we know about 
the equation AX = b is that A is invertible. We also know that solutions to AX = b 
come in three forms: exactly one solution, infinitely many solutions, and no solution. 
We just showed that if A is invertible, then AX = b has at least one solution. We 
showed that by setting x equal to ATIB, we have a solution. Is it possible that more 
solutions exist? 

No. Suppose we are told that a known vector Vis a solution to the equation AX = b; 
that is, we know that AV = B. We can repeat the above steps: 


AV =b 
ATAY = Atb 
i= Atb 
V= A1. 


This shows that all solutions to AX = b are exactly X = A-1b when A is invertible. We 
have just proved the following theorem. 


Theorem 8 Invertible Matrices and Solutions to Ax = b 


Let A be an invertible n x n matrix, and let b be anynx 1 


column vector. Then the equation AX = b has exactly one 
solution, namely 


X= Atb. 
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A corollary to this theorem is: If A is not invertible, then Ax = B does not have 
exactly one solution. It may have infinitely many solutions and it may have no solu- 
tion, and we would need to examine the reduced row echelon form of the augmented 
matrix [A b] to see which case applies. 

We demonstrate our theorem with an example. 


Example 57 Solve AX = b by computing X = ATID, where 

1 0 —3 —15 

A=j}-3 -4 10 and b= 57 

4 -5 -11 —46 

SOLUTION Without showing our steps, we compute 

94 15 —12 
A™t=| 7 1 -1 
31 5 —4 


We then find the solution to AX = b by computing A-1p: 


X=A'b 
94 15 —12 —15 
= 7 1 —1 57 
31 5 —4 —46 
—3 
= | —2 
4 
We can easily check our answer: 
1 Oo -3 3 —15 
—3 -4 10 —2 | = 57 
4 -5 -11 4 —46 


Knowing a matrix is invertible is incredibly useful.1® Among many other reasons, if 
you know A is invertible, then you know for sure that AX = b has a solution (as we just 
stated in Theorem 8). In the next section we’ll demonstrate many different properties 
of invertible matrices, including stating several different ways in which we know that 
a matrix is invertible. 


Exercises 2.6 


154 corollary is an idea that follows directly from a theorem 
16As odd as it may sound, knowing a matrix is invertible is useful; actually computing the inverse isn’t. 
This is discussed at the end of the next section. 


2.6 The Matrix Inverse 


In Exercises 1- 8, A matrix A is given. Find 5 -1 0 
A~' using Theorem 7, if it exists. 17. 7 7 1 
7 —2 -8 -1 
1 5 
t |5 a 1 -5 0 
g 18. | -2 15 4 
> f1 F) 4 -19 1 
1 -3 
a 25 —8 0 
[3 0 19. | -783 25 0 
3 0 7 | 48 —15 1 
2 5 1 0 0 
4 |3 A 20. |7 5 8 
A -2 -2 -3 
1 —3 
5 l> ¢ | |: 0 1 
a 21. 1 0 0 
z ; A 0 10 
L 0O 1 0 
1 o0 22. |1 0 0 
7. 16 | 001 
afe eee 
7 met S 33 4 1 -7 
In Exercises 9 — 28, a matrix A is given. Find 4 2 0 -1 
A™* using Key Idea 10, if it exists. 
Š 1 0 0 0 
—2 3 27 1 0 4 
2 ee | an | 18 01 4 
i T5 Z| 4 0 0 1 
` 9 2 —15 45 —3 4 
T 55 —164 15 —15 
a. |1 3] 25. | L215 640 -62 59 
[3 4 sae 12 0 1 
fection 7 | 1 0 2 8 
[5/3 7/3 s |0 2 0 0 
25 -10 —4 7 = E z. 
13. —18 7 3 
—6 2 1 0 0 1 0 
0 0 0 1 
2 3 4 27. jooo 
14. —3 6 9 0100 
—1 9 13 
1 0 0 0 
1 0 B 02 0 0 
15. |4 1 -7 2.1503 0 
20 7 —48 o 0 0 -4 
-4 1 5 In Exercises 29 — 36, a matrix A and a vector 
16. -5 1 9 b are given. Solve the equation Ax = b using 
—10 2 19 Theorem 8. 
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[3 5 -> [21 —34 
ee | al? abel b= | -159 
" ih 4 sfa —243 
' =ja —15|’ |77 
[9 70 PE 2 Se 
BA | a “ey lt P= |) 4 35. A= |-8 1 5j, 


$5 10 eal b= |~% 


1 2 12 
33. A=|0 1 6], =l 
-3 0 1 
—17 1 -6 0 
b= | -5 36.A=|0 1 ol, 
20 2 -8 1 
1 0 «8 —69 
34.A=|8 -2 -13], b=| 10 
12 -3 —20 —102 


2.7 Properties of the Matrix Inverse 


AS YOU READ; 


1. What does it mean to say that two statements are “equivalent?” 

2. T/F: If A is not invertible, then Ax = 0 could have no solutions. 

3. T/F: If A is not invertible, then Ax = b could have infinitely many solutions. 
4. What is the inverse of the inverse of A? 


5. T/F: Solving Ax = b using Gaussian elimination is faster than using the inverse 
of A. 


We ended the previous section by stating that invertible matrices are important. 
Since they are, in this section we study invertible matrices in two ways. First, we look 
at ways to tell whether or not a matrix is invertible, and second, we study properties 
of invertible matrices (that is, how they interact with other matrix operations). 

We start with collecting ways in which we know that a matrix is invertible. We 
actually already know the truth of this theorem from our work in the previous section, 
but it is good to list the following statements in one place. As we move through other 
sections, we'll add on to this theorem. 


Theorem 9 


2.7 Properties of the Matrix Inverse 


Invertible Matrix Theorem 


Let Abe ann xn matrix. The following statements are equiv- 
alent. 


(a) A is invertible. 
(b) There exists a matrix B such that BA = I. 
(c) There exists a matrix C such that AC = /. 


(d) The reduced row echelon form of A is /. 


(e) The equation AX = b has exactly one solution for ev- 
eryn x 1 vector b. 


(f) The equation AX = 0 has exactly one solution 
(namely, x = 0). 


Let’s make note of a few things about the Invertible Matrix Theorem. 


1. First, 


note that the theorem uses the phrase “the following statements are equiv- 


alent.” When two or more statements are equivalent, it means that the truth of 
any one of them implies that the rest are also true; if any one of the statements 
is false, then they are all false. So, for example, if we determined that the equa- 
tion Ax = Ohad exactly one solution (and A was ann x n matrix) then we would 
know that A was invertible, that Ax = bhad only one solution, that the reduced 
row echelon form of A was /, etc. 


2. Let’s go through each of the statements and see why we already knew they all 
said essentially the same thing. 


(a) 


(b) 


(c) 


(d) 


This simply states that A is invertible — that is, that there exists a matrix 
AT! such that AtA = AAT! = I. We'll go on to show why all the other 
statements basically tell us “A is invertible.” 


If we know that A is invertible, then we already know that there is a matrix 
B where BA = I. That is part of the definition of invertible. However, we 
can also “go the other way.” Recall from Theorem 5 that even if all we know 
is that there is a matrix B where BA = I, then we also know that AB = |. 
That is, we know that B is the inverse of A (and hence A is invertible). 


We use the same logic as in the previous statement to show why this is the 
same as “A is invertible.” 


If Ais invertible, we can find the inverse by using Key Idea 10 (which in turn 
depends on Theorem 5). The crux of Key Idea 10 is that the reduced row 
echelon form of A is I; if it is something else, we can’t find AT! (it doesn’t 
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exist). Knowing that A is invertible means that the reduced row echelon 
form of A is I. We can go the other way; if we know that the reduced row 
echelon form of A is I, then we can employ Key Idea 10 to find A~?, so A is 
invertible. 


(e) We know from Theorem 8 that if A is invertible, then given any vector b, 
Ax = bhas always has exactly one solution, namely x = ATIL. However, 
we can go the other way; let’s say we know that AX = b always has exactly 
solution. How can we conclude that A is invertible? 


Think about how we, up to this point, determined the solution to AX = 
b. We set up the augmented matrix [A b] and put it into reduced row 
echelon form. We know that getting the identity matrix on the left means 
that we had a unique solution (and not getting the identity means we either 
have no solution or infinitely many solutions). So getting / on the left means 
having a unique solution; having / on the left means that the reduced row 
echelon form of A is /, which we know from above is the same as A being 
invertible. 


(f) This is the same as the above; simply replace the vector b with the vector 
O. 


So we came up with a list of statements that are all equivalent to the statement 
“A is invertible.” Again, if we know that if any one of them is true (or false), then 
they are all true (or all false). 


Theorem 9 states formally that if A is invertible, then AX = b has exactly one solu- 
tion, namely A-1b. What if A is not invertible? What are the possibilities for solutions 
to Ax = b? 

We know that Ax = b cannot have exactly one solution; if it did, then by our the- 
orem it would be invertible. Recalling that linear equations have either one solution, 
infinitely many solutions, or no solution, we are left with the latter options when A is 
not invertible. This idea is important and so we'll state it again as a Key Idea. 


Key Idea 11 Solutions to AX = b and the Invertibility of A 


Consider the system of linear equations AX = b. 


1. IfA is invertible, then AX = b has exactly one solution, 
namely AT İb. 


. IfA is not invertible, then AX = b has either infinitely 
many solutions or no solution. 


In Theorem 9 we’ve come up with a list of ways in which we can tell whether or 
not a matrix is invertible. At the same time, we have come up with a list of properties 
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of invertible matrices — things we know that are true about them. (For instance, if we 
know that A is invertible, then we know that AX = b has only one solution.) 

We now go on to discover other properties of invertible matrices. Specifically, we 
want to find out how invertibility interacts with other matrix operations. For instance, 
if we know that A and B are invertible, what is the inverse of A+B? What is the inverse 
of AB? What is “the inverse of the inverse?” We’ll explore these questions through an 
example. 


Example 58 Let 


Find: 


A 3. (AB)! 5. (A+B)? 


2. B7! 4. (Att 6. (5A)7t 


In addition, try to find connections between each of the above. 


SOLUTION 


1. Computing AT! is straightforward; we’ll use Theorem 7. 


tji 2e 2] 


3/0 3 0 1 
2. We compute B~! in the same way as above. 
pte 1j|1 (0) 2 —1/2 0 
—2|-1 -2 1/2 1 


3. To compute (AB)~t, we first compute AB: 


We now apply Theorem 7 to find (AB)~?. 


(AB)! = È P an a 


=e |at —4 1/6 2/3 


4. To compute (A~)~+, we simply apply Theorem 7 to A?: 


ers -Ei 
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5. To compute (A + B)~1, we first compute A + B then apply Theorem 7: 


he ee ee 


(asey*= >] F =! 


Hence 


o;-1 1 
Our last expression is really nonsense; we know that if ad — bc = O, then the 


given matrix is not invertible. That is the case with A + B, so we conclude that 
A + Bis not invertible. 


6. To compute (5A)~1, we compute 5A and then apply Theorem 7. 


arda SD -ai EPE 2 


We now look for connections between A~?, Bt, (AB)~1, (A~?)~1 and (A+8B)~?. 


3. Is there some sort of relationship between (AB)~* and A~? and B™t? A first 


guess that seems plausible is (AB)~+ = A~1B-1, Is this true? Using our work 
from above, we have 


aipin be a | Be d = E a . 


0 1 1/2 1 1/2 1 


Obviously, this is not equal to (AB) ~+. Before we do some further guessing, let’s 
think about what the inverse of AB is supposed to do. The inverse — let’s call it 
C- is supposed to be a matrix such that 


(AB)C = C(AB) =1. 


In examining the expression (AB)C, we see that we want B to somehow “cancel” 
with C. What “cancels” B? An obvious answer is B71. This gives us a thought: 
perhaps we got the order of A~? and B~! wrong before. After all, we were hop- 
ing to find that 
ABA~'B-} = 1, 

but algebraically speaking, it is hard to cancel out these terms.” However, 
switching the order of A~? and B~? gives us some hope. Is (AB)? = B-1A~1? 
Let’s see. 


(AB)(B-*A~*) = A(BB~*)A~* (regrouping by the associative property) 


= AIA~* (BB * = 1) 
= AA~* (Al = A) 
= (a~ =) 


17 Recall that matrix multiplication is not commutative. 
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Thus it seems that (AB)~1 = B~1A~?. Let’s confirm this with our example ma- 


trices. 
eee EE Fe aer 
It worked! 


4. Is there some sort of connection between (A~*)~1 and A? The answer is pretty 
obvious: they are equal. The “inverse of the inverse” returns one to the original 
matrix. 


5. Is there some sort of relationship between (A+B)~*, A™t and B71? Certainly, if 
we were forced to make a guess without working any examples, we would guess 
that 


(A+B)? È AT! + B7. 


However, we saw that in our example, the matrix (A + B) isn’t even invertible. 
This pretty much kills any hope of a connection. 


6. Is there a connection between (5A)~t and A™t? Consider: 


e ge 
_1 ki i 
5| 0 1/5 
1 
=A 


Yes, there is a connection! 


Let’s summarize the results of this example. If A and B are both invertible matrices, 
then so is their product, AB. We demonstrated this with our example, and there is 
more to be said. Let’s suppose that A and B are n x n matrices, but we don’t yet know 
if they are invertible. If AB is invertible, then each of A and B are; if AB is not invertible, 
then A or B is also not invertible. 

In short, invertibility “works well” with matrix multiplication. However, we saw 
that it doesn’t work well with matrix addition. Knowing that A and B are invertible 
does not help us find the inverse of (A + B); in fact, the latter matrix may not even be 
invertible.18 

Let’s do one more example, then we’ll summarize the results of this section in a 
theorem. 


2 0 0 
Example 59 FindtheinverseofA=|0 3 0 
0 0 -7 


18The fact that invertibility works well with matrix multiplication should not come as a surprise. After 
all, saying that A is invertible makes a statement about the mulitiplicative properties of A. It says that | can 
multiply A with a special matrix to get /. Invertibility, in and of itself, says nothing about matrix addition, 
therefore we should not be too surprised that it doesn’t work well with it. 
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SOLUTION We'll find A~+ using Key Idea 10. 
20 0 100 os 100 1/2 0 (0) 
03 0 0 10 rref 010 0 1⁄3 (0) 
O0 0 -7 0 0 1 001 0 Oo -1/7 
Therefore 
1/2 0 (0) 
A™t=]| 0 13 0 


The matrix A in the previous example is a diagonal matrix: the only nonzero entries 
of A lie on the diagonal.’ The relationship between A and AT! in the above example 
seems pretty strong, and it holds true in general. We’ll state this and summarize the 
results of this section with the following theorem. 


Theorem 10 Properties of Invertible Matrices 


Let A and B ben x n invertible matrices. Then: 
. AB is invertible; (AB)~? = B™tAT t. 


. A~tis invertible; (A~*)~* = A. 


. nA is invertible for any nonzero scalar n; (nA)~? = 
14-1 
AE 


. If A is a diagonal matrix, with diagonal entries 
d,,d,---,d,, where none of the diagonal en- 
tries are 0, then AT! exists and is a diagonal ma- 
trix. Furthermore, the diagonal entries of A™t are 
1/d1,1/d2,--- ,1/dn. 


Furthermore, 


1. If a product AB is not invertible, then A or B is not in- 
vertible. 


2. If Aor B are not invertible, then AB is not invertible. 


We end this section with a comment about solving systems of equations “in real 
life”? Solving a system AX = b by computing AT tb seems pretty slick, so it would 


19We still haven’t formally defined diagonal, but the definition is rather visual so we risk it. See Definition 
24 on page 137 for more details. 
20Ves, real people do solve linear equations in real life. Not just mathematicians, but economists, engi- 
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make sense that this is the way it is normally done. However, in practice, this is rarely 
done. There are two main reasons why this is the case. 

First, computing A~* and ATIB is “expensive” in the sense that it takes up a lot of 
computing time. Certainly, our calculators have no trouble dealing with the 3 x 3 cases 
we often consider in this textbook, but in real life the matrices being considered are 
often very large (as in, hundreds of millions or rows and columns, or larger). Computing 
AT‘ alone is rather impractical, and we waste a lot of time if we come to find out 
that A~* does not exist. Even if we already know what A~? is, computing ATID is 
computationally expensive — Gaussian elimination is faster. 

Secondly, computing AT! using the method we’ve described often gives rise to 
numerical roundoff errors. Even though computers often do computations with an 
accuracy to more than 8 decimal places, after thousands of computations, roundoffs 
can cause big errors. (A “small” 1,000 x 1,000 matrix has 1, 000, 000 entries! That’s 
a lot of places to have roundoff errors accumulate!) It is not unheard of to have a 
computer compute AT! for a large matrix, and then immediately have it compute AA~* 
and not get the identity matrix.?! 

Therefore, in real life, solutions to AX = bare usually found using the methods we 
learned in Section 2.4. It turns out that even with all of our advances in mathematics, 
it is hard to beat the basic method that Gauss introduced a long time ago. 


Exercises 2.7 


In Exercises 1-4, matrices A and B are given. 6. A= [3 z 
Compute (AB)~* and B-*A~?. | 2 4 
_f1 2 f3 5 [o7 
HAS Jia al BS |< d TAS a z| 
[1 ‘| [ 7 | - 
2. A= BS 9 0 
3 4 2 1 = 
L L 8. A E d 
f2 5 J1 -1 
3. A= 3 gl’ B= 1 4 9. Find 2x2 matrices A and B that are each 
i 3 invertible, but A + B is not. 
Pie 2 4 Re 2 2 l 
|2 5 [6 5 10. Create a random 6 x 6 matrix A, then 


have a calculator or computer compute 
AA™+. Was the identity matrix returned 
exactly? Comment on your results. 


In Exercises 5 — 8, a2 x 2 matrix A is given. 
Compute A~’ and (A~*)~* using Theorem 7. 


5 A= -3 5 
e, T -2 11. Use a calculator or computer to com- 


neers, and scientists of all flavors regularly need to solve linear equations, and the matrices they use are 
often huge. 

Most people see matrices at work without thinking about it. Digital pictures are simply “rectangular 
arrays” of numbers representing colors — they are matrices of colors. Many of the standard image processing 
operations involve matrix operations. The author’s wife has a “7 megapixel” camera which creates pictures 
that are 3072 x 2304 in size, giving over 7 million pixels, and that isn’t even considered a “large” picture 
these days. 

21The result is usually very close, with the numbers on the diagonal close to 1 and the other entries near 
0. But it isn’t exactly the identity matrix. 
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pute AA~', where Was the identity matrix returned ex- 
1 2 3 4 actly? Comment on your results. 


a-|1 4 9 16 
~|1 8 27 64 
1 16 81 256 


2.8 Linear Independence 


AS YOU READ 


1. T/F: If Vand w are linearly dependent then vis a scalar multiple of w. 
2. T/F: If V2 = 2V1 — 4V; then the set {V1, V2, V3 } is linearly dependent. 
3. T/F: A set of 3 vectors in R* must be linearly independent. 


4. T/F: There is a set of 3 vectors in R? that is linearly independent. 


Motivation 


In mathematics, many important ideas begin as geometric questions. That is, we often 
start with concrete pictures and examples, which we then try to generalize to solve new 
and more complicated problems. As a first illustration of what we'll be getting at in this 


, : = 1 4 1 ; F 
section, consider the vectors Vv; = | 1 | and v2 = 1 | We'll use the convention 


that the first entry is the x-direction and the second entry is the y-direction. 


> 


Vi 


y Axis 


x Axis 


Figure 2.1: The vectors V1 and Vv. 


Geometrically, the vectors are pointing in different directions. That is, Vv; cannot 
be used to construct V2 in any meaningful way, in that V2 Æ cv, for any choice of c. 
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Figure 2.2: The vectors V1, V2 and V3. 


The concept of constructing one vector from one or more other vectors is funda- 
mental in matrix and linear algebra. The word “construct” in this context has a very 
specific meaning, embodied by the following definition: 


Definition 20 Linear Combinations of Vectors A /inear combination or su- 
perposition of vectors V1,V2,...,V, is a sum of the form 


C1V1 Ar C2V2 ar 28S CnVn (2.1) 


where c1, C2, . . . ,Cn are constants. 


We may also use summation notation to write a linear combination as 
n 
y CjVj. 
j=1 


Example 60 Let vı = | i | V = | = | and v3 = | P: | Verify that at 


least one of these vectors can be constructed as a linear combination of the other two. 


SOLUTION Geometrically, it appears as though all the vectors are pointing in 
different directions; see Figure 2.2. Yet we can check that 


»-[2]-[38 [28] 


So we can create V3 as a combination of the first two vectors. (We can in fact write any 
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of the three vectors as a linear combination of the other two.) 


While the above examples were relatively straightforward to analyze, we want to 
develop tools and ideas that will generalize to more complicated problems. How can 
we systematically determine when a vector can be expressed as a linear combination 
of other vectors? Even in R? it can be challenging to see by inspection if a given vector 
can be expressed as a linear combination of other vectors. 


Example 61 Let 
3 6 —9 
Vi = 4 i V2 = —2 s and V3 = 38 
5 —3 —20 


Verify that at least one of these vectors can be written as a linear combination of the 
other two. 


SOLUTION Although we’ll show a more general and methodical approach in 
a later subsection, in the present case we can simply try to write 


V3 = avı + bi, 


for some choice of a and b. This leads to three equations, 3a+6b = —9, 4a—2b = 38, 
and —5a — 3b = —20 for unknowns a and b. Surprisingly, a solution (unique) exists, 
namely a = 7, b = —5 (despite the fact that we have three equations in the two vari- 
ables a and b). Thus V3 = 7V, — 5V3. 


Example 62 Let 
3 6 —9 
Wy = 4 ; A = —2 ` and W3 = 38 
—5 -3 20 


Verify none of these vectors can be written as a linear combination of the other two. 


SOLUTION This is even less obvious. One approach is to try a combination 
of the form w; = aW + bw3 and attempt to solve for a and b; you find the system 
is inconsistent. A similar conclusion holds for combinations w2 = aw, + bw3 and 
W3 = aw, + bw». 


Definition of Linear Dependence 


Much of linear and matrix algebra is based on the idea of linear combinations of vec- 
tors. In particular, it will be useful to develop a strategy for determining when one 
vector can be expressed as a linear combination of a set of given vectors. Consider a 


2.8 Linear Independence 


fixed set S of vectors S = {V1, V2, . . . , Vn y and a vector V. We want to determine if it is 
possible to write 


V = CV, + C2V2 ++ + nV, 


for some choice of constants c1, . . . ,Cn. We will pose this as a system of linear equa- 
tions. Before we explore this more fully, we introduce an important concept. 


Definition 21 Linearly Dependent Vectors A set of vectors {V1, V2,...,Vn} 
is linearly dependent if there are constants c1, . . . , Cn so that 


CV + CV + +--+ CnVp 


with at least one of the c, Æ 0. 


The above definition means that the zero vector is a non-trivial linear combination of 
the vectors {V1, V2, . . . , Va}, where “non-trivial” means not all constants cx are zero. 


Example 63 Show that the vectors {V1, V2, V3 } with 


n=l pal and pal 
Im 1 f 2 —1 , g= 0 


are linearly dependent. 


SOLUTION One can verify that 
1 +%3 = 0. (2.2) 


Thus these vectors are linearly dependent. 


Note that equation (2.2) can be rearranged to express V; = —V2 — 2V3, or V) = 
—V1 — 2V3, or V3 = -4v — TA When a set of vectors is linearly dependent, it allows 
us to express one or more of the vectors as a linear combination of the others in the set. 


Example 64 Verify that set of vectors {t1, U2, U3, U4} with 
2 —1 3 1 
gali "IE: a a ; -3 
I= 3 , P 1 , gom 4 , 4— —10 
4 3 1 —33 


are linearly dependent. 
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SOLUTION This follows since 


12-—4-3+41 
6+0-3-3 
18 +4-— 12 — 10 
24 +12 — 3 — 33 


lI 
Ol 


6u, + 4u2 — 3u3 + U4 = 


Of course how one comes up with the coefficients 6, 4, —3, and 1 is not at all obvious. 


At this point, we can verify that a set of vectors are linearly dependent when we’re 
given the appropriate linear combination to check. However, it is not yet clear how 
we can generally determine if a set of vectors is linearly dependent. By using our skills 
with matrix arithmetic and systems of linear equations, we can develop a strategy to 
quickly solve the problem. 

Given a set of vectors {V1, V2, . . . , Vn } in R”, we want to determine if 


C1V1 + C2V2 +. + CnVn = 0 (2.3) 


has any non-trivial solutions. To analyze this, we write the vectors as 


V11 V21 Vja Vn1 

= V12 = V22 _ Vj2 . Vn2 

Vi = y , vV = ; ’ vV = , , Vn = ; 
Vim V2m Vim Vnm 


and rewrite the left side of (2.3) as 


C1V11 + C2V21 + ++ + CnVna 
C1V12 + C2V22 + +++ + CnVn2 


C1Vq + CaV2 +--+ + CnVn = 
C1Vim + C2V2m + +++ + CnhVnm 


This must be equal to the zero vector. Since each entry of the vectors must be equal, 
we have a system of m equations (one for each entry of the vector) in n unknowns (the 
constants c1, C2,- . . , Cn), 


C1V11 + CaV21 +++ Cnm =O 
C1V12 + C2V22 +*+ + CnVma = 
C1Vim + €2V2m + +++ + CnVnm =0 
We can now reduce the problem of linear dependence of the set of vectors {V1, ¥2,...,Vn} 


into determining whether there are non-trivial solutions to the homogeneous system 
of equations whose coefficients are determined by the vectors. We can represent this 
more compactly with an augmented matrix 


iad 


2.8 Linear Independence 


where the m x n matrix A is formed by taking the vectors V1, V2, . . . , V as the columns: 


Let’s revisit Example 60. 


Example 65 Determine if the vectors {V1, V2, V3 } with 


y= Vo = and V3 = 
1 |? 2 —1 |? a 0 
are linearly dependent. 


SOLUTION We form the matrix 


We now solve the homogeneous system of equations with coefficients represented by 
A. We may represent this with an augmented matrix 
1 1 -1 0 =} 1 0 —1/2 0 
A0| = rref 
[A o] es eoa 
From this we can see that the rank of the matrix is two since there are two pivots in 
the reduced row echelon form. Since there are three variables, c1, C2 and c3, there is 
a free variable in the solution. We can see that 
1 , F 1 , Lev 5 
=C3V1 + =C3V2 + C3V3 = 
ze + 7an 3V3 
for any choice of c3. In particular, if c3 = 1, we recover the solution in Example 60 
$01 + ih +3 = 0. 

Notice that if we chose c3 = —1 we'd see that — 4V; — 4V2 — V3 = 0. We can obtain 
another very similar linear combination for cz = 2 or c3 = 7 or c3 = 1.09945. There 
are infinitely many non-trivial linear combinations that equal the zero vector when the 
vectors are linearly dependent. 


Example 66 Determine if the vectors 
1 2 3 
—1 3 2 
v= 9 » v=}-4], v=|5 
1 4 5 
1 2 3 
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are linearly dependent. 


SOLUTION We form the 3 x 5 matrix whose columns are the vectors and 
reduce to row echelon form 
1 2 3 101 
—1 3 2 = O 1 1 
A= 9 -4 5 rref 00 0 
1 4 5 00 0 
1 2 3 00 0 
So that, if c3 = —1, we have Vv; + V — V = 0. Hence, the vectors are linearly inde- 
pendent. 
Example 67 Determine if the vectors 
1 2 0 
—1 3 2 
v=] 9 |, w=] 4l, vkv=]|1 
1 4 2 
1 2 1 
are linearly dependent. 
SOLUTION We form the 3 x 5 matrix whose columns are the vectors and re- 
duce to row echelon form 
1 2 0 1 0 0 
—1 3 2 ox 01 0 
A= 9 -4 1 rref 001 
1 4 2 00 0 
1 2 1 00 0 


So that, the only solution to AC = Ois if C1 = C2 = C3 = 0. Hence, these vectors are 
not linearly dependent. 


This deserves a name! 


Definition 22 A set of vectors {V1, V2, . . . , Vn } are linearly independent if 


the only solution to 


CVa + CoV. +--+ E Cnn = 0 


is the trivial solution. That is, c; = O for every constant cj, 
1<j<n. 
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In our last example, we saw the vectors weren’t linearly dependent. This means 
the vectors are linearly independent. In fact, many math textbooks define the idea of 
linear independence by saying the vectors are “not linearly dependent.” 


Example 68 Verify that the vectors 
1 2 (0) 
—1 3 2 
V = 9 » w=] 4|, v3=] 1 
1 4 2 
1 2 1 
are linearly independent. 
SOLUTION We saw in Example 67 that the only solution to c1V1+C2V2+C3V3 = 


Ois if cı = C2 = C3 = 0. Hence, these vectors are linearly independent. 


Example 69 Determine the value(s) of the constant k for which the vectors 
1 2 1 
w=al|1}], w=}]3], B=} 0 
1 2 k 


are linearly independent. 


SOLUTION As before, we form the matrix whose columns are the vectors. 
12 1 
A=|1 3 0 
1 2 k 


Here, we take a bit of care. Instead of trying to go directly to reduced row echelon 
form, by subtracting the first row from the second row and the third row we arrive at 


1 2 1 

O 1 —1 

0 0O k-1 
Provided that k — 1 4 0, we can continue on to reduced row echelon form. That is, if 
k Æ 1, the reduced row echelon form of A is the identity matrix, soc, = cp = cz = 0 


is the only solution to (2.3) and the vectors are linearly independent. 
If, on the other hand, k = 1, then the reduced row echelon form of A is 


And, if c3 = 1, we have —3¥, + V2 + V3 = 0 and the vectors are linearly dependent. 
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Connection to the Rank 


We note that in all examples above we are searching for solutions to the homogeneous 
equation (2.3), or in matrix form 
Ac = 0. 

We note that since homogeneous systems are always consistent, the trivial solution 
¢ = Ois always a solution, if there is a non-trivial solution there are infinitely many 
solutions. This tells us that there are free variables. This means, if we are clever, we 
don’t even have to solve the system, we just need to determine whether or not there 
are free variables. This is what the rank is good for! 


Theorem 11 Linear Independence and Rank 
Given a set of n vectors {V1, V2, . . . , Vn}, we form the matrix 


If the rank of A is n, there is only the trivial solution č = 0 
to the equation AC = O. If the rank of A is less than n, there 
are non-trivial solutions to the equation Ac = 0. 
Furthermore, the vectors {V1, V2, . . - , Vn} are linearly inde- 
pendent when the rank of A is n and linearly dependent 
when the rank of A is less than n. 


Example 70 Verify the vectors 
1 2 3 
—1 3 2 
v = 9 » w=] -4}], v3=]5 
1 4 5 
1 2 3 
are linearly dependent. 
SOLUTION In Example 65 We formed the 3 x 5 matrix whose columns are the 
vectors and reduced to row echelon form 
1 2 3 101 
—1 3 2 O 1 1 
— 
A= 9 —4 5 rref 0 0 0 
1 4 5 0 0 0 
1 2 3 00 0 
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Since there are 3 vectors, there are n = 3 unknowns in our equation. The rank of A is 
2 < 3, so there are free variables in the solution to (2.3). Hence, there are non-trivial 
solutions and the vectors are linearly dependent. 


Similarly, if we re-examine Example 67, we see that there are n = 3 vectors and 
the rank of A is also 3. Since there are then no free variables in the solution to (2.3), 
there is a unique solution c1 = cp = c3 = 0 and the vectors are linearly independent. 

Let’s bring this back to the motivating question. If a set of vectors {V1,V2,...,Vn} 
are linearly dependent, there is a non-trivial solution to 


Vp ots taste, = 0. (2.4) 


In particular, at least one constant cj # 0. Geometrically, this means we can con- 
struct V; as a linearly combination of the other vectors {V1,V2,...,Vj—1, Vj41,---,Va}- 
Specifically, if we rearrange (2.4) we have 


CjVj = —C1V1 C2V2 res Cj—1Vj—1 — Cj+1Vj+1 — tt CnVn. 
Since cj Æ 0, we can write 


> a, C2, Cj—1 > Cj+1 > Cn > 
j= A V1 V2 kat ma Vj—1 È Vj+1 suet Vn. 
J J J J J 


In a sense, the vector V; is thus “redundant.” There is no linear combination of Vi,..., Vn 
that cannot be constructed by using just {V1, V2,...,Vj-1, Vj41,; - - - , Vn} (omitting V;.) 

What this means is that if we have a set S = {V1, . . . , Vn } and we can identity the 
linear dependencies in the set—the redundant vectors—we can omit these and possi- 
bly simplify future computations, since we can work more economically with a smaller 
set of vectors. You should plan to take MA371 Linear Algebra in a future term! 


Exercises 2.8 


In Exercises 1 — 11, state whether or not the 4. 
given vectors are linearly dependent or inde- 1 0 2 
pendent. O;,}; 17,4 41 
1 0 2 
1. 
1 3 
Bae . 
1 2 2 
3 »,{| 24y,] 1 
5 1 =) -1 1 2 
[abla] 
6. 
3. 1 0 0 
1 0 1 0 1 1 
o;,/ 1], )] 1 Oo};’| OF’ ] 2 
0 0 2 1 0 1 
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10. 


11. 


12. 


13. 
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1 —4 0 
12 1 0 
nm |? 5 >| 0 
1 19 0 
1 0 1 1 
(gy dU: en em 3 
0 0 2 —2 
1 0 0 8 
0 1 0 6 
0 : 0 |’ 1 ; 7 
0 0 0 5 
1 3 1 8 
1 1 0 6 
—1 f 1 : 1 } 7 
0 0 0 0 
1 0 0 1 
1 1 1 0 
1 f 1 ? 1 ; 3 
1 —1 0 1 


How must two vectors Vı and Vz be re- 
lated if they are linearly dependent? 


Given the vectors 


1 —2 


(a) Are the vectors V1, V2 and V3 lin- 
early independent or linearly de- 
pendent? 


(b) Compute the rank of A, the ma- 
trix whose column vectors are 
V1, V2, V3. 


14. 


15. 


16. 


18. 


(c) Solve the system of equations 
Xi = 2X2 — X% = 0 


X2 +x3=0 
—xX, + 3x2 + 2x3 =0 


(d) (More open-ended question.) Ex- 
plain how the previous questions 
are related. Can you explain how 
the approach to solving each is 
the same? or different? 


Find all values of b for which the vectors 


are linearly dependent. 


Find all values of b for which the vectors 


1 b 2 
a Rn ee Wee 
—1 1 2 


are linearly dependent. 


Given a set of 4 vectors in R’. Is the set 
linearly independent or linearly depen- 
dent? Explain. 


. Given a set of 3 vectors in R?. Is the set 


linearly independent or linearly depen- 
dent? Explain. 


Given a set of 13 vectors in R®. Is the 
set linearly independent or linearly de- 
pendent? Explain. 


| Given a set of vectors in R”, is there a 


maximum number of vectors that can 
be linearly independent? Explain. 


. Thereisno minimum number of vectors 


that are linearly dependent. Give an ex- 
ample of a set of two vectors in R? that 
are linearly dependent. Can you give an 
example of a set of one vector that is 
linearly independent? 


OPERATIONS ON MATRICES 


In the previous chapter we learned about matrix arithmetic: adding, subtracting, and 
multiplying matrices, finding inverses, and multiplying by scalars. In this chapter we 
learn about some operations that we perform on matrices. We can think of them as 
functions: you input a matrix, and you get something back. One of these operations, 
the transpose, will return another matrix. With the other operations, the trace and 
the determinant, we input matrices and get numbers in return, an idea that is different 
than what we have seen before. 


3.1 The Matrix Transpose 
1. T/F: If A isa3 x 5 matrix, then A’ will be a 5 x 3 matrix. 
2. Where are there zeros in an upper triangular matrix? 
3. T/F: A matrix is symmetric if it doesn’t change when you take its transpose. 
4. What is the transpose of the transpose of A? 


5. Give 2 other terms to describe symmetric matrices besides “interesting.” 


We jump right in with a definition. 


Chapter 3 Operations on Matrices 
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Definition 23 Transpose 


Let A be an m x n matrix. The tranpsose of A, denoted A’, 
is the n x m matrix whose columns are the respective rows 
of A. 


Examples will make this definition clear. 


Example 71 Find the transpose of A = Lap : 
4 5 6 
SOLUTION Note that A is a 2 x 3 matrix, so A’ will be a 3 x 2 matrix. By the 


definition, the first column of A’ is the first row of A; the second column of A’ is the 
second row of A. Therefore, 


1 4 
A=|2 5 
3 6 
Example 72 Find the transpose of the following matrices. 
7 2 9 1 1 10 -2 
A=|2 -1 3 0| B=/3 -5 7 c=[1 -1 7 8 3] 
5 3 0 11 4 2 -3 
SOLUTION We find each transpose using the definition without explanation. 


Make note of the dimensions of the original matrix and the dimensions of its transpose. 


1 
7 2 = -5 

1 3 4 —1 

s= 7 B'= | 10 -5 2 c=|7 

oO ii —2 7 -3 8 

3 


Notice that with matrix B, when we took the transpose, the diagonal did not change. 
We can see what the diagonal is below where we rewrite B and B’ with the diagonal in 
bold. We’ll follow this by a definition of what we mean by “the diagonal of a matrix,” 
along with a few other related definitions. 


1 10 -2 1 3 4 
B=|3 -5 7 B'= | 10 -5 2 
4 2 -3 —2 7 -3 


It is probably pretty clear why we call those entries “the diagonal.” Here is the 
formal definition. 


3.1 The Matrix Transpose 


Definition 24 The Diagonal, a Diagonal Matrix, Triangular Matrices 


Let A be an m x n matrix. The diagonal of A consists of the 
entries d11, Q22,... Of A. 


A diagonal matrix is an n x n matrix in which the only 
nonzero entries lie on the diagonal. 


An upper (lower) triangular matrix is a matrix in which any 
nonzero entries lie on or above (below) the diagonal. 


Example 73 Consider the matrices A, B, C and l4, as well as their transposes, 
where 
1 2 3 3 0 0 : F : 
A=]|0 4 5 B=|O 7 0 E= 006 
0 0 6 0 0 1 0o00 


Identify the diagonal of each matrix, and state whether each matrix is diagonal, upper 
triangular, lower triangular, or none of the above. 


SOLUTION We first compute the transpose of each matrix. 
1 0 0 3 0 0 100 0 
A’=|2 40| B'=|0 7 0 cl=|2 4 00 
3 5 6 00 -1 3 5 60 


Note that /} = l4. 

The diagonals of A and A’ are the same, consisting of the entries 1, 4 and 6. The 
diagonals of B and B’ are also the same, consisting of the entries 3, 7 and —1. Finally, 
the diagonals of C and C’ are the same, consisting of the entries 1, 4 and 6. 

The matrix A is upper triangular; the only nonzero entries lie on or above the diag- 
onal. Likewise, A’ is lower triangular. 

The matrix B is diagonal. By their definitions, we can also see that B is both upper 
and lower triangular. Likewise, /, is diagonal, as well as upper and lower triangular. 

Finally, Cis upper triangular, with C’ being lower triangular. 


Make note of the definitions of diagonal and triangular matrices. We specify that 
a diagonal matrix must be square, but triangular matrices don’t have to be. (“Most” 
of the time, however, the ones we study are.) Also, as we mentioned before in the 
example, by definition a diagonal matrix is also both upper and lower triangular. Fi- 
nally, notice that by definition, the transpose of an upper triangular matrix is a lower 
triangular matrix, and vice-versa. 
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There are many questions to probe concerning the transpose operations.? The 
first set of questions we'll investigate involve the matrix arithmetic we learned from 
last chapter. We do this investigation by way of examples, and then summarize what 
we have learned at the end. 


Example 74 Let 


Find A’ + B’ and (A +B)’. 


SOLUTION We note that 
1 4 1 3 
A'=|2 5] andB’=|]2 -1 
3 6 1 0 
Therefore 
1 4 1 3 
A'+B'=|2 5]/4+]2 -1 
3 6 1 0 
2 7 
=14 4 
4 6 
Also, 
1 2 3] [f1 2 ayy" 
Pa 
(AB) =, 5 JBE = al) 
fle 4 aly 
E 7 4 6 
2 7 
=14 4l. 
4 6 


It looks like “the sum of the transposes is the transpose of the sum”? This should 
lead us to wonder how the transpose works with multiplication. 


Example 75 Let 


1 2 1 2 -1 
A= and B = : 
3 4 1 0O 1 
Remember, this is what mathematicians do. We learn something new, and then we ask lots of questions 
about it. Often the first questions we ask are along the lines of “How does this new thing relate to the old 
things | already know about?” 


138 2This is kind of fun to say, especially when said fast. Regardless of how fast we say it, we should think 
about this statement. The “is” represents “equals.” The stuff before “is” equals the stuff afterwards. 


3.1 The Matrix Transpose 
Find (AB)', A'B" and BTA’. 


SOLUTION We first note that 


2 4 A 
Find (AB)': 
7 _f{1 2]]1 2 -1 
ar-a allaa al) 
ffs 2 ayy" 
(7 6 1 
3 7 
=|2 6 
1 1 
Now find A’B’: 


= Not defined! 


So we can’t compute A’B’. Let’s finish by computing B'AT: 


We may have suspected that (AB)’ = ATB". We saw that this wasn’t the case, 
though — and not only was it not equal, the second product wasn’t even defined! Oddly 
enough, though, we saw that (AB)’ = B'AT. ? To help understand why this is true, 
look back at the work above and confirm the steps of each multiplication. 

We have one more arithmetic operation to look at: the inverse. 


Example 76 Let 


3Then again, maybe this isn’t all that “odd.” It is reminiscent of the fact that, when invertible, (AB)~1 = 
B-14-1, 
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Find (A~*)! and (A’)—?. 
SOLUTION We first find A~! and A’: 


1 f4 -7 r [2 1 
A de | anaa’ = |? al: 


Finding (A~*)': 


Finding (A‘)~?: 


It seems that “the inverse of the transpose is the transpose of the inverse.”4 


We have just looked at some examples of how the transpose operation interacts 
with matrix arithmetic operations.” We now give a theorem that tells us that what we 
saw wasn’t a coincidence, but rather is always true. 


Theorem 12 Properties of the Matrix Transpose 


Let A and B be matrices where the following operations are 
defined. Then: 
1. (A +B)" =A’ +B’ and (A — B)" =A’ — B' 


kA n 


( 
- ( 
- ( 
. As 
- 


We included in the theorem two ideas we didn’t discuss already. First, that (kKA)™ = 


4 Again, we should think about this statement. The part before “is” states that we take the transpose of 
a matrix, then find the inverse. The part after “is” states that we find the inverse of the matrix, then take 
the transpose. Since these two statements are linked by an “is,” they are equal. 

5These examples don’t prove anything, other than it worked in specific examples. 
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kA". This is probably obvious. It doesn’t matter when you multiply a matrix by a scalar 
when dealing with transposes. 

The second “new” item is that (A’)’ = A. That is, if we take the transpose of a 
matrix, then take its transpose again, what do we have? The original matrix. 

Now that we know some properties of the transpose operation, we are tempted 
to play around with it and see what happens. For instance, if A is an m x n matrix, we 
know that A’ is an n x m matrix. So no matter what matrix A we start with, we can 
always perform the multiplication AA’ (and also AA) and the result is a square matrix! 

Another thing to ask ourselves as we “play around” with the transpose: suppose A 
is a Square matrix. Is there anything special about A + A’? The following example has 
us try out these ideas. 


Example 77 Let 
2 1 3 
A=]2 -1 1 
1 0 1 
Find AA’, A + A7 and A — A’. 
SOLUTION Finding AA’: 
2 1 3/72 2 1 
AA = |2 -1 1 1 —1 0 
1 0 3 1 1 
14 6 5 
=|6 4 3 
5 3 2 
Finding A +A’: 
2 1 3 2 2 1 
A+A’=|]2 -1 1] +]1 -1 0 
10 1 3 1 1 
2 3 4 
=/3 —2 1 
4 1 2 
Finding A — A’: 
2 1 3 2 2 1 
A-A'T=1]2 -1 1 1 -1 0 
101 3 1 1 
0o —1 2 
=/1 01 
—2 -1 0 
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Let’s look at the matrices we’ve formed in this example. First, consider AA’. Some- 
thing seems to be nice about this matrix — look at the location of the 6’s, the 5’s and 
the 3’s. More precisely, let’s look at the transpose of AA’. We should notice that if we 
take the transpose of this matrix, we have the very same matrix. That is, 


T 


We’ll formally define this in a moment, but a matrix that is equal to its transpose 
is called symmetric. 

Look at the next part of the example; what do we notice about A +A’? We should 
see that it, too, is symmetric. Finally, consider the last part of the example: do we 
notice anything about A — A‘? 

We should immediately notice that it is not symmetric, although it does seem 
“close.” Instead of it being equal to its transpose, we notice that this matrix is the 
opposite of its transpose. We call this type of matrix skew symmetric.© We formally 
define these matrices here. 


Definition 25 Symmetric and Skew Symmetric Matrices 


A matrix A is symmetric if A’ = A. 


A matrix A is skew symmetric if A’ = —A. 


Note that in order for a matrix to be either symmetric or skew symmetric, it must 
be square. 

So why was AA’ symmetric in our previous example? Did we just luck out?’ Let’s 
take the transpose of AA’ and see what happens. 


(Aa) = (A (A)! transpose multiplication rule 


= AA! (aY =A 


We have just proved that no matter what matrix A we start with, the matrix AA’ will 
be symmetric. Nothing in our string of equalities even demanded that A be a square 
matrix; it is always true. 

We can do a similar proof to show that as long as A is square, A--A’ is a symmetric 
matrix.2 We’ll instead show here that if A is a square matrix, then A — A’ is skew 


6Some mathematicians use the term antisymmetric 
7 Of course not. 
8Why do we say that A has to be square? 


3.1 The Matrix Transpose 


symmetric. 


(A == Ay = A = (Arye transpose subtraction rule 


So we took the transpose of A — A’ and we got — (A — A’); this is the definition of 
being skew symmetric. 

We’ll take what we learned from Example 77 and put it in a box. (We’ve already 
proved most of this is true; the rest we leave to solve in the Exercises.) 


Theorem 13 Symmetric and Skew Symmetric Matrices 


. Given any matrix A, the matrices AA” and A’A are sym- 
metric. 


. Let A be a square matrix. The matrix A + A’ is sym- 
metric. 


. Let A be a square matrix. The matrix A — A’ is skew 
symmetric. 


Why do we care about the transpose of a matrix? Why do we care about symmetric 
matrices? 

There are two answers that each answer both of these questions. First, we are in- 
terested in the tranpose of a matrix and symmetric matrices because they are interest- 
ing.One particularly interesting thing about symmetric and skew symmetric matrices 
is this: consider the sum of (A +A’) and (A — A’): 


(A+A‘) + (A—A’) = 2A. 


This gives us an idea: if we were to multiply both sides of this equation by z, then the 
right hand side would just be A. This means that 


A= LA +A‘) + P(A —A’). 
oe E 
symmetric skew symmetric 
That is, any matrix A can be written as the sum of a symmetric and skew symmetric 
matrix. That’s interesting. 

The second reason we care about them is that they are very useful and important in 
various areas of mathematics. The transpose of a matrix turns out to be an important 
operation; symmetric matrices have many nice properties that make solving certain 
types of problems possible. 
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Most of this text focuses on the preliminaries of matrix algebra, and the actual uses 
are beyond our current scope. One easy to describe example is curve fitting. Suppose 
we are given a large set of data points that, when plotted, look roughly quadratic. How 
do we find the quadratic that “best fits” this data? The solution can be found using 
matrix algebra, and specifically a matrix called the pseudoinverse. If A is a matrix, the 
pseudoinverse of A is the matrix At = (ATA)~1AT (assuming that the inverse exists). 
We aren’t going to worry about what all the above means; just notice that it has a cool 
sounding name and the transpose appears twice. 

In the next section we’ll learn about the trace, another operation that can be per- 
formed on a matrix that is relatively simple to compute but can lead to some deep 
results. 


Exercises 3.1 


In Exercises 1- 24, a matrix A is given. Find 4 0 -2 
A’; make note if A is upper/lower triangular, 13. 0 2 3 
diagonal, symmetric and/or skew symmetric. —2 3 6 
i [-7 4 | | o 3 -2 
“| 4 —-6 14. 3 -4 1 
3 rs :] —2 1 0 
"|-7 8 2 -5 -3 
By aes, 
[0 9 
13 3 0O —6 1 
4 -3 1 | 16 6 0O 4 
> —1 -4 0 
-5 -9 
5. | 3 1 E a Se 
—10 -8 17. 5 —4 —10 
—6 6 9 
—2 10 
6. | i7 | BOs S. 
9 2 18. | -2 -7 0 
E 4 2 5 
7 4 -7 -4 -9 
"|-9 6 3 | ea A 
Š 19. 0o -3 5 
3 3 -10 0 | 0 0 -3 
—10 -2 -3 1 
6 -7 2 6 
9. [-7 -8 2 -3] 20 fe -8 -1 0 
10. [-9 8 2 -7] 0 0 1 -7 
-9 4 10 1 0 0 
11. 6 -3 -7 21. |O 2 0 
-8 1 -1 0 0 -1 
4 -5 2 6 —4 -5 
Tte 5 9 22. |-4 0 2 
9 2 3 -5 2 -2 
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0 1 -2 0 0 0 
23. |—1 0 4 24. |O 0 0 
2 —4 0 0 0 0 


3.2 The Matrix Trace 


AS YOU READ 


1. T/F: We only compute the trace of square matrices. 


2. T/F: One can tell if a matrix is invertible by computing the trace. 


In the previous section, we learned about an operation we can peform on matrices, 
namely the transpose. Given a matrix A, we can “find the transpose of A,” which is 
another matrix. In this section we learn about a new operation called the trace. It is 
a different type of operation than the transpose. Given a matrix A, we can “find the 
trace of A,” which is not a matrix but rather a number. We formally define it here. 


Definition 26 The Trace 


Let A be ann x n matrix. The trace of A, denoted tr(A), is 
the sum of the diagonal elements of A. That is, 


tr(A) = a11 + a22 +++ + Ann. 


This seems like a simple definition, and it really is. Just to make sure it is clear, let’s 
practice. 


Example 78 Find the trace of A, B, Cand l4, where 
1 2 0 
asi peis 3 8 1 and c= l; : a 
—2 7 —5 
SOLUTION To find the trace of A, note that the diagonal elements of A are 1 


and 4. Therefore, tr(A) =1+4=5. 
We see that the diagonal elements of B are 1, 8 and -5, so tr(B)= 1 +8 — 5 = 4. 
The matrix C is not a square matrix, and our definition states that we must start 
with a square matrix. Therefore tr(C) is not defined. 
Finally, the diagonal of /4 consists of four 1s. Therefore tr(/,) = 4. 
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Now that we have defined the trace of a matrix, we should think like mathemati- 
cians and ask some questions. The first questions that should pop into our minds 
should be along the lines of “How does the trace work with other matrix operations?” 
We should think about how the trace works with matrix addition, scalar multiplication, 
matrix multiplication, matrix inverses, and the transpose. 


We’ll give a theorem that will formally tell us what is true in a moment, but first 
let’s play with two sample matrices and see if we can see what will happen. Let 


2, A 3 2 0 1 
A=|]2 0 -1] andB=}]-1 2 0 
3 1 3 0 2 -1 


2 1 3 2 0 1 
tr(A +B) =tr 2 0 -1/;+]-1 2 O 
3 -1 3 0 2 1 
4 1 4 
=tr 1 2 -1 
3 1 2 
=8 


So we notice that tr(A + B) = tr(A) + tr(B). This probably isn’t a coincidence. 


How does the trace work with scalar multiplication? If we multiply A by 4, then the 
diagonal elements will be 8, O and 12, so tr(4A) = 20. Is it a coincidence that this is 4 
times the trace of A? 


Let’s move on to matrix multiplication. How will the trace of AB relate to the traces 
of A and B? Let’s see: 


2 1 3 2 0 1 
tr(AB)=tr| |2 0 —-1 —1 2 0 
3 -1 3 0 2 -1 
3 8 -1 
=tr 4 -2 3 
7 4 0 
=1 


It isn’t exactly clear what the relationship is among tr(A), tr(B) and tr(AB). Before 


Recall that we asked a similar question once we learned about the transpose. 


3.2 The Matrix Trace 


moving on, let’s find tr(BA): 


2 0 1 2 1 3 
tr(BA)=tr | |—-1 2 0 2 0 -1 
0 2 1 3 -1 3 

7 1 9 

=tr 2 —1 -5 

1 1 -5 


We notice that tr(AB) = tr(BA). Is this coincidental? 
How are the traces of A and A~' related? We compute A~? and find that 


1/17 6/17 1/17 
At = | 9/17 3/17 -8/17 
2/17 —5/17 2/17 


Therefore tr(A—*) = 6/17. Again, the relationship isn’t clear.?° 

Finally, let’s see how the trace is related to the transpose. We actually don’t have 
to formally compute anything. Recall from the previous section that the diagonals 
of A and A’ are identical; therefore, tr(A) = tr(A’). That, we know for sure, isn’t a 
coincidence. 

We now formally state what equalities are true when considering the interaction 
of the trace with other matrix operations. 


Theorem 14 Properties of the Matrix Trace 


Let A and B ben x n matrices. Then: 
. tr(A + B) = tr(A) + tr(B) 
. tr(A — B) = tr(A) — tr(B) 
. tr(kA) = k-tr(A) 


. tr(AB) = tr(BA) 


. tr(A’) = tr(A) 


One of the key things to note here is what this theorem does not say. It says nothing 
about how the trace relates to inverses. The reason for the silence in these areas is 
that there simply is not a relationship. 


10something to think about: we know that not all square matrices are invertible. Would we be able to 
tell just by the trace? That seems unlikely. 
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We end this section by again wondering why anyone would care about the trace of 
matrix. One reason mathematicians are interested in it is that it can give a measure- 
ment of the “size” of a matrix. 

Consider the following 2 x 2 matrices: 


1 -2 6 7 
A= È 1 and B = E eA 
These matrices have the same trace, yet B clearly has bigger elements in it. So how 
can we use the trace to determine a “size” of these matrices? We can consider tr(A’A) 


and tr(B’B). 
vaa =er([ i, ale a) 


ta) 


=7 


Our concern is not how to interpret what this “size” measurement means, but 
rather to demonstrate that the trace (along with the transpose) can be used to give 
(perhaps useful) information about a matrix.’ 


11There are many different measurements of a matrix size. In this text, we just refer to its dimensions. 
Some measurements of size refer the magnitude of the elements in the matrix. The next section describes 
yet another measurement of matrix size. 
12This example brings to light many interesting ideas that we'll flesh out just a little bit here. 
1. Notice that the elements of A are 1, —2, 1 and 1. Add the squares of these numbers: 1? + (—2)? + 
1? +1? = 7 = tr(A'A). 
Notice that the elements of B are 6, 7, 11 and -4. Add the squares of these numbers: 6? + 72 + 
112 + (—4)? = 222 = tr(B’B). 
Can you see why this is true? When looking at multiplying ATA, focus only on where the elements 
on the diagonal come from since they are the only ones that matter when taking the trace. 


2. You can confirm on your own that regardless of the dimensions of A, tr(A’A) = tr(AA’). To see why 
this is true, consider the previous point. (Recall also that ATA and AA’ are always square, regardless 
of the dimensions of A.) 


3. Mathematicians are actually more interested in \/tr(A’A) than just tr(A’A). The reason for this is a 
bit complicated; the short answer is that “it works better.” The reason “it works better” is related to 
the Pythagorean Theorem, all of all things. If we know that the legs of a right triangle have length 
a and b, we are more interested in V'a? + b? than just a? + b*. Of course, this explanation raises 
more questions than it answers; our goal here is just to whet your appetite and get you to do some 
more reading. A Numerical Linear Algebra book would be a good place to start. 


Exercises 3.2 


3.3 The Determinant 


In Exercises 1- 15, find the trace of the given 


matrix. 
1 —5 
a 
—3 —10 
ae 
7 5 
TE 
—6 0 
i E J 
—4 1 1 
5 |—2 0 0 
—1 —2 -5 
0 -3 1 
6. 5 -5 5 
—4 1 0 
—2 -3 5 
7. 5 2 0 
—1 -3 1 
4 2 —1 
8. |-4 1 
0 -5 5 
2 6 4 
3: E 8 = 
6 5 
10. | 2 10 
3 3 
—10 6 -7 
—2 1 6 
Il: (0) 4 —4 
-3 -9 3 


3.3 The Determinant 


AS YOU READ 


12. 


13. 


14. 


15. 


A matrix A that is skew symmetric. 


In Exercises 16 — 19, verify Theorem 14 by: 


1. 


16. 


17. 


18. 


19. 


Showing that tr(A)+tr(B) = tr(A + B) 
and 


. Showing that tr(AB) = tr(BA). 


sai | @=| = S| 


0 -8 —4 5 
asfi ap rs[a] 
-8 —10 10 
A=] 10 5 -6 

—10 1 3 
—10 -4 -3 
B=|-4 -5 4 
3 7 3 
-10 7 5 
A=| 7 7 =5 
8 -9 2 
-3 -4 9 
B=|4 -1 -9 
-7 -8 10 


1. T/F: The determinant of a matrix is always positive. 


2. T/F: To compute the determinant of a 3 x 3 matrix, one needs to compute the 


determinants of 3 2 x 2 matrices. 


3. Give an example of a 2 x 2 matrix with a determinant of 3. 
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In this chapter so far we’ve learned about the transpose (an operation on a ma- 
trix that returns another matrix) and the trace (an operation on a square matrix that 
returns a number). In this section we’ll learn another operation on square matrices 
that returns a number, called the determinant. We give a pseudo-definition of the 
determinant here. 


The determinant of an n x n matrix A is a number, denoted 
det(A), that is determined by A. 


That definition isn’t meant to explain everything; it just gets us started by making 
us realize that the determinant is a number. The determinant is kind of a tricky thing 
to define. Once you know and understand it, it isn’t that hard, but getting started is a 
bit complicated.1? We start simply; we define the determinant for 2 x 2 matrices. 


Definition 27 Determinant of 2 x 2 Matrices 


a=|? ae 


Let 


The determinant of A, denoted by 


det (A) or 


We’ve seen the expression ad — bc before. In Section 2.6, we saw that a 2 x 2 matrix 
A has inverse 
1 d -—b 
ad—bc|—-c a 


as long as ad — bc Æ 0; otherwise, the inverse does not exist. We can rephrase the 
above statement now: If det(A)4 0, then 


= aay [eo 


A brief word about the notation: notice that we can refer to the determinant by us- 
ing what looks like absolute value bars around the entries of a matrix. We discussed at 
the end of the last section the idea of measuring the “size” of a matrix, and mentioned 
that there are many different ways to measure size. The determinant is one such way. 
Just as the absolute value of a number measures its size (and ignores its sign), the de- 
terminant of a matrix is a measurement of the size of the matrix. (Be careful, though: 
det(A) can be negative!) 

Let’s practice. 


13|t’s similar to learning to ride a bike. The riding itself isn’t hard, it is getting started that’s difficult. 


3.3 The Determinant 


Example 79 Find the determinant of A, B and C where 
1 2 3 -1 1 -3 
al E 7 | anac= | 7, i k 
SOLUTION Finding the determinant of A: 


det (A) = ee | 
= 1(4) — 28) 
= —2. 


Similar computations show that det (B) = 3(7) — (—1)(2) = 23 and det (C) = 
1(6) — (—3)(—2) = 0. 


Finding the determinant of a 2 x 2 matrix is pretty straightforward. It is natural to 
ask next “How do we compute the determinant of matrices that are not 2 x 2?” We 
first need to define some terms.“ 

Definition 28 Matrix Minor, Cofactor 
Let A be an n x n matrix. The i,j minor of A, denoted A; j, is 
the determinant of the (n — 1) x (n — 1) matrix formed by 


deleting the i” row and jt column of A. 


The i, j-cofactor of A is the number 


Cy = (—1) MA i. 


Notice that this definition makes reference to taking the determinant of a matrix, while 
we haven’t yet defined what the determinant is beyond 2 x 2 matrices. We recognize 
this problem, and we’ll see how far we can go before it becomes an issue. 

Examples will help. 


Example 80 Let 
1 ee : : ; ; 
A=|4 5 6] andB= 
739 —1 9 -4 6 
1 1 1 1 


Find A13, A3,2, B2,1, B4,3 and their respective cofactors. 


14This is the standard definition of these two terms, although slight variations exist. 
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SOLUTION To compute the minor A; 3, we remove the first row and third col- 
umn of A then take the determinant. 
1 2 3 42 3 45 
a=|45 6/>]45 6/=|% 5] 
7 8 9 7 8 9 
4 5 


ma =| 7 8 |=32-35 = -3. 
The corresponding cofactor, C; 3, is 
C13 = (—1)'*7Ar3 = (—1)*(-3) = -3. 
The minor A3 2 is found by removing the third row and second column of A then 


taking the determinant. 


1 2 3 1 
A=|4 5 6|» l4 
789 7 
1 
4 


The corresponding cofactor, C3,2, is 


G2 = (—1}Ħ?A2 = (—1)9(-6) = 6. 


The minor Bz, is found by removing the second row and first column of B then 
taking the determinant. 


1 2 0 8 4 2 0 8 
-l3 5 7 2| |as 7 af |3 535 
-1 9 -4 6 49 -4 6 so 
1 1 1 1 #111 
2 0 8j. 
Bi=|9 —4 6|=? 
1 1 1 


We’re a bit stuck. We don’t know how to find the determinate of this 3 x 3 matrix. 
We’ll come back to this later. The corresponding cofactor is 


C1 = (—1) HB3 1 = —B21, 


whatever this number happens to be. 
The minor B43 is found by removing the fourth row and third column of B then 
taking the determinant. 


1 2 0 


N 
e ONO 
I o| 
e UW 
e OUN 
PANO 
Ha Oo) N OO 


5 
—1 9 <2 
1 
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28|. 
Baa =| -3 5 2 |=? 
9 6 


Again, we’re stuck. We won't be able to fully compute Cy 3; all we know so far is that 
Ca3 = (—1)**7Ba3 = (—1)Ba3. 


Once we learn how to compute determinates for matrices larger than 2 x 2 we can 
come back and finish this exercise. 


In our previous example we ran into a bit of trouble. By our definition, in order 
to compute a minor of ann x n matrix we needed to compute the determinant of a 
(n —1) x (n — 1) matrix. This was fine when we started with a 3 x 3 matrix, but when 
we got up to a 4 x 4 matrix (and larger) we run into trouble. 

We are almost ready to define the determinant for any square matrix; we need 
one last definition. 


Definition 29 Cofactor Expansion 
Let A be ann x n matrix. 
The cofactor expansion of A along the it? row is the sum 


ai aCi 1 + a;i, 2Ci,2 + +++ + AinCin- 


The cofactor expansion of A down the j* column is the sum 


ar Cry + G2 jC2j ++ + An jCn,j- 


The notation of this definition might be a little intimidating, so let’s look at an ex- 
ample. 


Example 81 Let 


D 
II 
Aere 
œo an N 
Oooow 


Find the cofactor expansions along the second row and down the first column. 


SOLUTION By the definition, the cofactor expansion along the second row is 
the sum 


a2,1C2,1 + 02,2C2,2 + A2323. 


(Be sure to compare the above line to the definition of cofactor expansion, and see 
how the “i” in the definition is replaced by “2” here.) 
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We’ll find each cofactor and then compute the sum. 


G 1= (-1)?"" 


2 3 we removed the second row and 
8 9 = (—1)(—6) =6 first column of A to compute the 


minor 


second column of A to compute 
the minor 


1 3 we removed the second row and 
5 g AWH 


G 3= (yrs 


1 2 we removed the second row and 
= (—1)(—6) =6 third column of A to compute the 
7 8 minor 


Thus the cofactor expansion along the second row is 


a2,1C2,1 + a2,2C2,2 + a2,3C2,3 = 4(6) + 5(—12) + 6(6) 
= 24 — 60 + 36 
=0 


At the moment, we don’t know what to do with this cofactor expansion; we’ve just 
successfully found it. 


We move on to find the cofactor expansion down the first column. By the defini- 
tion, this sum is 


a1,1C1,1 + a2,1C2,1 + Q3,103,1- 


(Again, compare this to the above definition and see how we replaced the “j” with “1.”) 
We find each cofactor: 


5 6 . . 
1+1 = — d the first d first 
Cia (—1) 8 9 | = (1)(—3) =o Cee oe 
C21 = (-1)?*" : : | = (—1)(-6) =6 ( we computed this cofactor above ) 
2 3 F . 
_ 341 2 _ d the third d first 
C34 = (—1) 5 6 | = (1)(—3) = a 


The cofactor expansion down the first column is 


a1,1C1,1 + 02,1021 + 03103,1 = 1(—3) + 4(6) + 7(—3) 
= -3 +24— 21 
=0 


Is it a coincidence that both cofactor expansions were 0? We’ll answer that in a 
while. 
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This section is entitled “The Determinant,” yet we don’t know how to compute it 
yet except for 2 x 2 matrices. We finally define it now. 


Definition 30 The Determinant 


The determinant of an n x n matrix A, denoted det(A) or |A], 
is anumber given by the following: 


e ifAisa1 x 1 matrix A = [a], then det (A) = a. 


e ifAisa2 x 2 matrix 


then det (A) = ad — bc. 


e ifAisann xn matrix, where n > 2, then det(A) is the 
number found by taking the cofactor expansion along 
the first row of A. That is, 


det (A) = a1,1C1,1 + 1,2C1,2 +--+ +1 nCipn. 


Notice that in order to compute the determinant of ann x n matrix, we need to com- 
pute the determinants of n (n — 1) x (n— 1) matrices. This can be a lot of work. We’ll 
later learn how to shorten some of this. First, let’s practice. 


Example 82 Find the determinant of 
12 3 
A=|4 5 6 
7 8 9 
SOLUTION Notice that this is the matrix from Example 81. The cofactor ex- 


pansion along the first row is 
det (A) = a1 1C1,1 + @1,2C1.2 + a1 31,3. 


We’ll compute each cofactor first then take the appropriate sum. 


Cia = (—1) HA1 Cra = (—1) A2 C13 = (-1)'*7A13 
5 6 4 6 4 5 
ae 9 =D a sol 8 
= 45 — 48 = (—1)(36 — 42) =32 -35 
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Therefore the determinant of A is 


det (A) = 1(—3) + 2(6) + 3(—3) = 0. 


Example 83 Find the determinant of 
3 6 7 
A=|]0 2 —1 
3 -1 1 
SOLUTION We’ll compute each cofactor first then find the determinant. 
C11 = (—1)"""Ais Qa2= (—1)*"" Ais a3= (—1)*"Ai3 
2 -1 0 -1 0 2 
ea | Se] eee 
=2-1 = (—1)(0 + 3) =0-6 
=1 =-3 = 
Thus the determinant is 
det (A) = 3(1) + 6(—3) + 7(—6) = —57. 
Example 84 Find the determinant of 
1 2 1 2 
-1 2 3 4 
ae 8 5 -3 1 
5 9 -6 3 
SOLUTION This, quite frankly, will take quite a bit of work. In order to compute 


this determinant, we need to compute 4 minors, each of which requires finding the 
determinant of a3 x 3 matrix! Complaining won’t get us any closer to the solution, * 
so let’s get started. We first compute the cofactors: 


Cia = (1) A1 


2 3 4 
t te the det i t 

ee ee a 

9 -6 3 

3 1 5 1 5 —3 

ae _4)\i4+1 . (1)14+2 -(4)143 
=2. (1| 7? [+3 (yt? | 2 3 [+4 ey |e =| 
= 2(—3) + 3(—6) + 4(-3) 


15But it might make us feel a little better. Glance ahead: do you see how much work we have to do?!? 
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-1 3 4 
t te the det i t 
ie ee on 
5 = 
=3 1 8 1 8 -3 
PEN E = sfn AIFI . (—_1\1+2 . (—1\143 
= (-1) [aay] “3 icol ecel a 


——————$=[————————————————————— SS 


the determinate of the 3 x 3 matrix 
= (-1) [(—1)(—3) + 3(—19) + 4(—33)] 
= 186 


—1 2 4 
t te the determinant 
= 1. 8 ; ate compare ttie aiii 
5 
5 1 8 1 8 5 
= (—1).(—1)!*1 faye as 
= (ayo ] i| i 3 fee coe] 5] 
= (—1)(6) + 2(—19) + 4(47) 
= 144 
Gia = (—1)1™ A1 4 
—1 2 3 
t te the determinant 
FE a oo 
5 PE: 
2 8 -3 8 5 
ae ae 2: _4\1+1 eee: panes 
= [-9-cay] § <3 [+2-com| 3 |4s-co] #3] 


————— eee 


the determinate of the 3 x 3 matrix 
= (—1) [(—1)(—3) + 2(33) + 3(47)] 
= —210 


We’ve computed our four cofactors. All that is left is to compute the cofactor ex- 
pansion. 


det (A) = 1(—36) + 2(186) + 1(144) + 2(—210) = 60. 


As a way of “visualizing” this, let’s write out the cofactor expansion again but in- 
cluding the matrices in their place. 


157 


Chapter 3 Operations on Matrices 


158 


det (A) = a4,1C11 + @1,2C1,2 + a1,3C1,3 + a1,4C1,4 


2 3 4 —1 3 4 
=1(-1)?} 5 -3 1|+2(-1)| 8 -3 1 
9 —6 3 5 —6 3 
<— m n—[—>_)-— 
= —36 = —186 
+ 
—1 2 4 -1 2 3 
1(-1)*| 8 5 1]|+2(-1)?| 8 5 -3 
5 9 3 5 9 -6 
—— amm n—-_|_-_ _—_—__- 
= 144 = 210 


= 60 


That certainly took a while; it required more than 50 multiplications (we didn’t 
count the additions). To compute the determinant of a5 x 5 matrix, we’ll need to 
compute the determinants of five 4 x 4 matrices, meaning that we’ll need over 250 
multiplications! Not only is this a lot of work, but there are just too many ways to make 
silly mistakes.2© There are some tricks to make this job easier, but regardless we see 
the need to employ technology. Even then, technology quickly bogs down. A 25 x 25 
matrix is considered “small” by today’s standards,’ but it is essentially impossible for 
a computer to compute its determinant by only using cofactor expansion; it too needs 
to employ “tricks.” 

In the next section we will learn some of these tricks as we learn some of the 
properties of the determinant. Right now, let’s review the essentials of what we have 
learned. 


1. The determinant of a square matrix is a number that is determined by the matrix. 


2. We find the determinant by computing the cofactor expansion along the first 
row. 


3. To compute the determinant of ann x n matrix, we need to compute n deter- 
minants of (n — 1) x (n — 1) matrices. 


Exercises 3.3 


In Exercises 1- 8, find the determinant of the 2. | 6 A 
2 x 2 matrix. 


10 7 —1 -7 
a | ae A 
16The author made three when the above example was originally typed. 


1 It is common for mathematicians, scientists and engineers to consider linear systems with thousands 
of equations and variables. 


3.3 The Determinant 


i [10° =1 1-2 1 
` L —4 7 16. 5 5 4 
4 0 0 
8 10 | 
5 
[2 —3 1-4 1 
: T 10 —10 17 0 3 0 
l-10 0 1 2 2 
[1 =3 3 -1 0 
J3 4 18. | -3 0 —4 
z 0 -1 -4 
—4 -5 
8: | -1 =| -5 0 -4 
In Exercises 9 — 12, a matrix A is given. 19. 7 i a 
(a) Construct the submatrices used to 
compute the minors A; 1, A1,2 and A1,3. 1 0 0 
(b) Find the cofactors C1,1, C1,2, and C13. 20. O 1 0 
—1 1 1 
—7 —3 10 
9 3 7 6 Oo oO -1 —1 
1 6 10 1 1 0 1 
5 9 6 al; 1 1 —1 0 
7 ~ —1 0 1 0 
10 —10 -6 8 
0 -3 -2 —1 0 0 -1 
28 D >» |10 0 1 
11. | -3 3 10 peip. AD 
Tarua o 1 0 -1 -1 
-6 —4 6 —5 1 o o0 
12. | -8 0 0 23, |73 3 2 5 
—10 8 -1 ' |2 4 3 4 
5 4 -3 3 
In Exercises 13 — 24, find the determinant 
of the given matrix using cofactor expansion 2 = 4 4 
along the first row. y <3. 3 2 
24. 
3 2 3 0 4 —5 1 
13. | -6 1 00 Z2 -5 —2 -5 
oa mi ag 25. Let Abe a2 x 2 matrix; 
8 —9 -2 
14. | -9 9 -7 aa Ae 
5 —1 9 
-4 3 4 Show why det(A) = ad—bc by comput- 
15. | -4 -5 3 ing the cofactor expansion of A along 
3 -4 5 the first row. 
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3.4 Properties of the Determinant 


AS YOU READ 


1. Having the choice to compute the determinant of a matrix using cofactor expan- 
sion along any row or column is most useful when there are lots of what in a row 
or column? 


2. Which elementary row operation does not change the determinant of a matrix? 


3. T/F: When computers are used to compute the determinant of a matrix, cofactor 
expansion is rarely used. 


In the previous section we learned how to compute the determinant. In this sec- 
tion we learn some of the properties of the determinant, and this will allow us to 
compute determinants more easily. In the next section we will see one application 
of determinants. 

We start with a theorem that gives us more freedom when computing determi- 
nants. 


Theorem 15 Cofactor Expansion Along Any Row or Column 


Let A be ann x n matrix. The determinant of A can be com- 
puted using cofactor expansion along any row or column of 
A. 


We alluded to this fact way back after Example 81. We had just learned what cofac- 
tor expansion was and we practiced along the second row and down the third column. 
Later, we found the determinant of this matrix by computing the cofactor expansion 
along the first row. In all three cases, we got the number 0. This wasn’t a coincidence. 
The above theorem states that all three expansions were actually computing the de- 
terminant. 

How does this help us? By giving us freedom to choose any row or column to use 
for the expansion, we can choose a row or column that looks “most appealing.” This 
usually means “it has lots of zeros.” We demonstrate this principle below. 


Example 85 Find the determinant of 
| 1 2 0 | 
o] 2 -3 0 5 
=] 7 2 3 8|’ 
—4 1 0 2 


3.4 Properties of the Determinant 


SOLUTION Our first reaction may well be “Oh no! Not another 4 x 4 deter- 
minant!” However, we can use cofactor expansion along any row or column that we 
choose. The third column looks great; it has lots of zeros in it. The cofactor expansion 
along this column is 


det (A) = a1,3C1,3 + a2,3Co,3 + a3,3C3.3 + a4,3Ca3 
= 0: C13 +0- C23 +3: C33 +0: C43 


The wonderful thing here is that three of our cofactors are multiplied by 0. We 
won't bother computing them since they will not contribute to the determinant. Thus 


det (A) =3. C33 


1 2 9 
=3-(-1)**.-| 2 -3 5 
—4 1 2 
— we computed the determinant of the 3 x 3 matrix 
=3. (—147) ( without showing our work; it is —147 ) 


Wow. That was a lot simpler than computing all that we did in Example 84. Of 
course, in that example, we didn’t really have any shortcuts that we could have em- 
ployed. 


Example 86 Find the determinant of 
1 2 3 4 5 
06 7 8 9 
A=1]/]0 0 10 11 12 
00 O 13 14 
00 0 0 15 
SOLUTION At first glance, we think “I don’t want to find the determinant of 


a5 x 5 matrix!” However, using our newfound knowledge, we see that things are not 
that bad. In fact, this problem is very easy. 

What row or column should we choose to find the determinant along? There are 
two obvious choices: the first column or the last row. Both have 4 zeros in them. We 
choose the first column.1® We omit most of the cofactor expansion, since most of it is 
just 0: 


6 7 8 9 
0 10 11 12 
det(A)=1-(—1)"™-) 9 o 13 14 
00 0 15 


18We do not choose this because it is the better choice; both options are good. We simply had to make 
a choice. 
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Similarly, this determinant is not bad to compute; we again choose to use cofac- 
tor expansion along the first column. Note: technically, this cofactor expansion is 
6 - (—1)***A11; we are going to drop the (—1)**? terms from here on out in this 
example (it will show up a lot...). 


10 11 12 
det(A)=1-6-| 0 13 14 
0 0 15 


You can probably can see a trend. We'll finish out the steps without explaining 
each one. 


0 15 
=1-6-10-13-15 
= 11700 


det(A) = 1:6-10- n d 


We see that the final determinant is the product of the diagonal entries. This works 
for any triangular matrix (and since diagonal matrices are triangular, it works for diag- 
onal matrices as well). This is an important enough idea that we'll put it into a box. 


Key Idea 12 The Determinant of Triangular Matrices 


The determinant of a triangular matrix is the product of its 
diagonal elements. 


It is now again time to start thinking like a mathematician. Remember, mathe- 
maticians see something new and often ask “How does this relate to things I already 
know?” So now we ask, “If we change a matrix in some way, how is it’s determinant 
changed?” 

The standard way that we change matrices is through elementary row operations. 
If we perform an elementary row operation on a matrix, how will the determinant of 
the new matrix compare to the determinant of the original matrix? 

Let’s experiment first and then we'll officially state what happens. 


Example 87 Let 


Let B be formed from A by doing one of the following elementary row operations: 
1. 2R +R > R2 


2. 5R > Ri 


3.4 Properties of the Determinant 


3. Ri © R2 


Find det(A) as well as det(B) for each of the row operations above. 


SOLUTION It is straightforward to compute det (A) = —2. 
Let B be formed by performing the row operation in 1) on A; thus 


It is clear that det (B) = —2, the same as det(A). 
Now let B be formed by performing the elementary row operation in 2) on A; that 


5 10 
B= È | 
We can see that det (B) = —10, which is 5 - det (A). 
Finally, let B be formed by the third row operation given; swap the two rows of A. 


We see that 
3 4 
s=[3 2] 


and that det (B) = 2, which is (—1) - det (A). 


is, 


We’ve seen in the above example that there seems to be a relationship between 
the determinants of matrices “before and after” being changed by elementary row 
operations. Certainly, one example isn’t enough to base a theory on, and we have not 
proved anything yet. Regardless, the following theorem is true. 


Theorem 16 The Determinant and Elementary Row Operations 


Let A be ann x n matrix and let B be formed by performing 
one elementary row operation on A. 


1. If Bis formed from A by adding a scalar multiple of 


one row to another, then det (B) = det (A). 


. If Bis formed from A by multiplying one row of A by a 
scalar k, then det (B) = k - det (A). 


. If Bis formed from A by interchanging two rows of A, 
then det (B) = —det (A). 


Let’s put this theorem to use in an example. 


163 


Chapter 3 Operations on Matrices 


164 


Example 88 Let 


D 

II 
eOr 
eeN 
PRR 


Compute det(A), then find the determinants of the following matrices by inspection 
using Theorem 16. 


1 1 1 1 2 1 1 —1 -2 
B=|1 2 1 C=|O 1 1 D= |O 1 1 
O 1 1 7 7 7 1 1 1 
SOLUTION Computing det(A) by cofactor expansion down the first column or 


along the second row seems like the best choice, utilizing the one zero in the matrix. 
We can quickly confirm that det (A) = 1. 

To compute det(B), notice that the rows of A were rearranged to form B. There are 
different ways to describe what happened; saying R; + R, was followed by R; © R3 
produces B from A. Since there were two row swaps, det (B) = (—1)(—1)det (A) = 
det (A) = 1. 


Notice that C is formed from A by multiplying the third row by 7. Thus det (C) = 
7 - det (A) =7. 


It takes a little thought, but we can form D from A by the operation —3R2+R1 — R1. 
This type of elementary row operation does not change determinants, so det (D) = 
det (A). 


Let’s continue to think like mathematicians; mathematicians tend to remember 
“problems” they’ve encountered in the past,and when they learn something new, in 
the backs of their minds they try to apply their new knowledge to solve their old prob- 
lem. 

What “problem” did we recently uncover? We stated in the last chapter that even 
computers could not compute the determinant of large matrices with cofactor expan- 
sion. How then can we compute the determinant of large matrices? 

We just learned two interesting and useful facts about matrix determinants. First, 
the determinant of a triangular matrix is easy to compute: just multiply the diagonal 
elements. Secondly, we know how elementary row operations affect the determinant. 
Put these two ideas together: given any square matrix, we can use elementary row op- 
erations to put the matrix in triangular form,*? find the determinant of the new matrix 
(which is easy), and then adjust that number by recalling what elementary operations 
we performed. Let’s practice this. 
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3.4 Properties of the Determinant 


Example 89 Find the determinant of A by first putting A into a triangular form, 
where 
2 4 -2 
A=j;-1 -2 5 
3 2 1 
SOLUTION In putting A into a triangular form, we need not worry about get- 


ting leading 1s, but it does tend to make our life easier as we work out a problem by 
hand. So let’s scale the first row by 1/2: 


1 2 —1 
ER, = Ry —1 —2 5 
3 2 1 
Now let’s get Os below this leading 1: 
Ri +R, >R ao S 
1 2 2 
o o0 4 
—3R R R 
1 + R3 > R3 0-4 4 


We can finish in one step; by interchanging rows 2 and 3 we'll have our matrix in 
triangular form. 


1 2 -1 
R2 © R3 Oo —4 4 
0 o0 4 


Let’s name this last matrix B. The determinant of B is easy to compute as it is 
triangular; det (B) = —16. We can use this to find det(A). 
Recall the steps we used to transform A into B. They are: 


ÍR, > Ry 
Ri + Ro > Ro 
—3R, + R3 > R3 
R2 © R3 


The first operation multiplied a row of A by F, This means that the resulting matrix 
had a determinant that was i the determinant of A. 

The next two operations did not affect the determinant at all. The last operation, 
the row swap, changed the sign. Combining these effects, we know that 


1 
—16 = det (B) = (—1) 5 det (A). 
Solving for det (A) we have that det (A) = 32. 


In practice, we don’t need to keep track of operations where we add multiples 
of one row to another; they simply do not affect the determinant. Also, in practice, 
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these steps are carried out by a computer, and computers don’t care about leading 1s. 
Therefore, row scaling operations are rarely used. The only things to keep track of are 
row swaps, and even then all we care about are the number of row swaps. An odd 
number of row swaps means that the original determinant has the opposite sign of 
the triangular form matrix; an even number of row swaps means they have the same 
determinant. 


Let’s practice this again. 


Example 90 The matrix B was formed from A using the following elementary 
row operations, though not necessarily in this order. Find det(A). 


2Rı > R 
12 3 nO 
o 3R3 —> R3 
B= |O 4 5 
006 Ri & R2 
6R, + Ro > Ro 
SOLUTION It is easy to compute det (B) = 24. In looking at our list of elemen- 


tary row operations, we see that only the first three have an effect on the determinant. 
Therefore 


24 = det (B) = 2- Ż - (—1) - det (A) 


and hence 


det (A) = —36. 


In the previous example, we may have been tempted to “rebuild” A using the ele- 
mentary row operations and then computing the determinant. This can be done, but 
in general it is a bad idea; it takes too much work and it is too easy to make a mistake. 

Let’s think some more like a mathematician. How does the determinant work with 
other matrix operations that we know? Specifically, how does the determinant in- 
teract with matrix addition, scalar multiplication, matrix multiplication, the transpose 
and the trace? We’ll again do an example to get an idea of what is going on, then give 
a theorem to state what is true. 


Example 91 Let 


1 2 2 1 
a=|3 A and a= |3 aE 


Find the determinants of the matrices A, B, A + B, 3A, AB, A’, A71, and compare the 
determinant of these matrices to their trace. 


SOLUTION We can quickly compute that det (A) = —2 and that det (B) = 7. 


3.4 Properties of the Determinant 


sae) —aee([? 2]-[? 2]) 
Een 
1 


II 


It’s tough to find a connection between det(A — B), det(A) and det(B). 


det (34) =| ae | 
= —18 


We can figure this one out; multiplying one row of A by 3 increases the determinant 
by a factor of 3; doing it again (and hence multiplying both rows by 3) increases the 
determinant again by a factor of 3. Therefore det (3A) = 3 -3- det (A), or 3? - A. 


see) =<er([ 2] [2 2]) 


ee: n 
18 23 
= —14 


This one seems clear; det (AB) = det (A) det (B). 


N e 


TE | 


RW 


Obviously det (A’) = det (A); is this always going to be the case? If we think about 
it, we can see that the cofactor expansion along the first row of A will give us the same 
result as the cofactor expansion along the first column of A". 


pe I. 
de )=| Hr aA | 
=1=3/2 
SAP 


20This can be a bit tricky to think out in your head. Try it with a 3x3 matrix A and see how it works. All 
the 2 x 2 submatrices that are created in A’ are the transpose of those found in A; this doesn’t matter since 
it is easy to see that the determinant isn’t affected by the transpose in a 2 x 2 matrix. 
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It seems as though 
1 


det (A~*) = ay 


We end by remarking that there seems to be no connection whatsoever between 
the trace of a matrix and its determinant. We leave it to the reader to compute the 
trace for some of the above matrices and confirm this statement. 


We now state a theorem which will confirm our conjectures from the previous 
example. 
Theorem 17 Determinant Properties 


Let A and B be n x n matrices and let k be a scalar. The 
following are true: 


. det (kA) = K” - det (A) 


. det (A7) = det (A) 


. det (AB) = det (A) det (B) 
. If Ais invertible, then 


1 


det (A~*) = EA 


. A matrix A is invertible if and only if det (A) # 0. 


This last statement of the above theorem is significant: what happens if det (A) = 
0? It seems that det (A+) =“1/0”, which is undefined. There actually isn’t a problem 
here; it turns out that if det (A) = 0, then A is not invertible (hence part 5 of Theorem 
17). This allows us to add on to our Invertible Matrix Theorem. 


Theorem 18 Invertible Matrix Theorem 


Let A be ann xn matrix. The following statements are equiv- 
alent. 


(a) Ais invertible. 


(g) det (A) #0. 
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This new addition to the Invertible Matrix Theorem is very useful; we’ll refer back 
to it in Chapter 4 when we discuss eigenvalues. 


We end this section with a shortcut for computing the determinants of 3 x 3 ma- 
trices. However, here is an important message first: 


THIS ONLY WORKS FOR 3 x 3 MATRICES!!! 


Consider the matrix A: 


NAB 
o Uu N 
WO aD Ww 


We can compute its determinant using cofactor expansion as we did in Example 82. 
Once one becomes proficient at this method, computing the determinant of a 3 x 3 
isn’t all that hard. A method many find easier, though, starts with rewriting the matrix 
without the brackets, and repeating the first and second columns at the end as shown 
below. 


1 2 3 1 2 
45 64 5 
7 8 9 7 8 


In this 3 x 5 array of numbers, there are 3 full “upper left to lower right” diagonals, 
and 3 full “upper right to lower left” diagonals, as shown below with the arrows. 


105 48 72 45 84 96 


The numbers that appear at the ends of each of the arrows are computed by mul- 
tiplying the numbers found along the arrows. For instance, the 105 comes from mul- 
tiplying 3-5-7 = 105. The determinant is found by adding the numbers on the right, 
and subtracting the sum of the numbers on the left. That is, 


det (A) = (45 + 84 + 96) — (105 + 48 + 72) = 0. 


To help remind ourselves of this shortcut, we’ll make it into a Key Idea. 
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Key Idea 13 3 x 3 Determinant Shortcut 


Let A be a3 x 3 matrix. Create a 3 x 5 array by repeating 
the first 2 columns and consider the products of the 3 “right 
hand” diagonals and 3 “left hand” diagonals as shown pre- 


viously. Then 


det (A) = “(the sum of the right hand numbers) 
— (the sum of the left hand numbers)”. 


We’ll practice once more in the context of an example. 


Example 92 Find the determinant of A using the previously described shortcut, 
where 
1 3 9 
A=j|;—2 3 4 
—-5 7 2 
SOLUTION Rewriting the first 2 columns, drawing the proper diagonals, and 


multiplying, we get: 


—135 28—12 6 —60 —126 


Summing the numbers on the right and subtracting the sum of the numbers on the 
left, we get 


det (A) = (6 — 60 — 126) — (—135 + 28 — 12) = —61. 


In the next section we’ll see how the determinant can be used to solve systems of 
linear equations. 


Exercises 3.4 


In Exercises 1 — 14, find the determinant =4 4 `A 
of the given matrix using cofactor expansion 2. (0) Oo 3 
along any row or column you choose. =2 —2 —-1 
1 2 3 —4 1 1 

1. | -5 0 3 3. (0) 0 0 
4 0 6 —1 —2 -5 


wu l 
N 
nN | 
w 
ow 
es | 


10. 


11. 


12. 


13. 


NONrRRPR 
WONO 


2 1 1 1 1 
4 1 2 0 2 
14. |O 0 1 0 0 
1 3 2 0 3 
5 05 0 4 


In Exercises 15 — 18, a matrix M and det(M) 
are given. Matrices A, B and C are formed by 


3.4 Properties of the Determinant 


performing operations on M. Determine the 
determinants of A, B and C using Theorems 
16 and 17, and indicate the operations used 


to form A, B and C. 


0o 3 5 
15. M=|3 1 0, 
= a | 
det(M) = —41 
0 3 5 
(a) A= |-2 -4 -1 
3 1 0 
0 3 5 
(b) B=|3 1 0 
8 16 4 
3 4 5 
() c=] 3 1 o0 
= ee aa 
9 7 8 
16. M=]1 3 7], 
6 3 3 
det(M) = 45. 


9 6 
() C=|7 3 
8 3 


9 7 8 
(b) B=|1 3 7 
1 
0 
o 
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5 4 0 g- [° ° 
18. M=]|7 9 3], Sja 4 
1 3 9 


In Exercises 23 — 30, find the determinant of 


det(M) = 120 
et(m) the given matrix using Key Idea 13. 
13 9 
(a) A=|7 9 3 a we OR 
5 4 0 23. | —6 1 —10 
-8 -9 -9 
5 4 0 
(b) B= |]14 18 6 8 -9 -2 
3 9 27 24. |-9 9 —7 
-5 -4 0 SA EEA 2 
(c) C= | -7 -9 -3 -4 3 —4 
P =e 25. | -4 -5 3 
In Exercises 19 — 22, matrices A and B are 3 —4 5 
given. Verify part 3 of Theorem 17 by com- 
puting det(A), det(B) and det(AB). 1 -2 1 
a 26. |5 5 4 
_ 4 0 0 
19. A F ah 
fo —4 1 -4 1 
B=!, 3 27. lo 3 0 
3 1 2 2 
3 -1 
20. A= | j 
|4 1 | 3 -1 0 
T -4 -1 28. | -3 0 —4 
B=! 5 3 0o -1 4 
—4 4 -5 0 —4 
21. A= 
| 5 =) 29. | 2 4 -1 
f= = -5 0 -4 
PE 3 l 
5 —3 
= 1 0 0 
—3 -1 30. Oo 1 0 
RAS | 2 a -1 1 1 
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3.5  Cramer’s Rule 


AS YOU READ 


1. T/F: Cramer’s Rule is another method to compute the determinant of a matrix. 


2. T/F: Cramer’s Rule is often used because it is more efficient than Gaussian elim- 
ination. 


3. Mathematicians use what word to describe the connections between seemingly 
unrelated ideas? 


In the previous sections we have learned about the determinant, but we haven’t 
given a really good reason why we would want to compute it.2 This section shows one 
application of the determinant: solving systems of linear equations. We introduce this 
idea in terms of a theorem, then we will practice. 


Theorem 19 Cramer’s Rule 


Let A be an n x n matrix with det (A) 4 O and let b be an 
n x 1 column vector. Then the linear system 


Ax =b 


has solution 


~ det (A) ’ 


where Ai(b) is the matrix formed by replacing the i*” column 
of A with b. 


Let’s do an example. 


Example 93 Use Cramer’s Rule to solve the linear system AX = B where 
1 5 -3 —36 
A=|1 4 2 and b = | —11 
2 -1 0 7 
SOLUTION We first compute the determinant of A to see if we can apply Cramer’s 


21The closest we came to motivation is that if det (A) = 0, then we know that A is not invertible. But it 
seems that there may be easier ways to check. 
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Rule. 
1 5 -3 
det(A)=|1 4 2 |=49. 
2 -1 0 


Since det (A) # 0, we can apply Cramer’s Rule. Following Theorem 19, we com- 
pute det (4:16), det (42(6)) and det (as). 


-36 5 -3 
det (41(6)) =| -11 4 2 |=49. 
7 1 0 


(We used a bold font to show where b replaced the first column of A.) 


1 -36 -3 
det (4.(6)) =| 1 -11 2 |=-—245. 
2 7 (0) 
1 5 —36 
det (43(6)) =|1 4 -11 |=196. 
2 -1 7 
Therefore we can compute x: 
det (4:(6)) 49 
— == = 1 
n= det(A) «49 
det (42(b))  _pa5 
X2 == — = 5 
det (A) 49 
det (43(5)) 196 
= = = 4 
3 det (A) 49 
Therefore 
X1 1 
x= X2 = —5 
X3 4 
Let’s do another example. 
Example 94 Use Cramer’s Rule to solve the linear system AX = b where 
1 2 Pi —1 
ael J and B= |". 
SOLUTION The determinant of A is —2, so we can apply Cramer’s Rule. 
> —1 2 
det (4:(6)) = | a | Zab 
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det (42(6)) = | : ři | =4 
Therefore 
det (41(6)) wf 
= = = 3 
“= det(A) 2 
det (42(6)) í 
=—=-2 
2 det (A) Z7 
and 


We learned in Section 3.4 that when considering a linear system AX = b where A 
is square, if det (A) 4 0 then A is invertible and Ax = b has exactly one solution. We 
also stated in Key Idea 11 that if det (A) = 0, then A is not invertible and so therefore 
either Ax = b has no solution or infinite solutions. Our method of figuring out which 
of these cases applied was to form the augmented matrix [A b] , put it into reduced 
row echelon form, and then interpret the results. 

Cramer’s Rule specifies that det (A) 4 0 (so we are guaranteed a solution). When 
det (A) = 0 we are not able to discern whether infinite solutions or no solution exists 
for a given vector b. Cramer’s Rule is only applicable to the case when exactly one 
solution exists. 


We end this section with a practical consideration. We have mentioned before 
that finding determinants is a computationally intensive operation. To solve a linear 
system with 3 equations and 3 unknowns, we need to compute 4 determinants. Just 
think: with 10 equations and 10 unknowns, we’d need to compute 11 really hard de- 
terminants of 10 x 10 matrices! That is a lot of work! 

The upshot of this is that Cramer’s Rule makes for a poor choice in solving nu- 
merical linear systems. It simply is not done in practice; it is hard to beat Gaussian 
elimination.22 

So why include it? Because its truth is amazing. The determinant is a very strange 
operation; it produces a number in a very odd way. It should seem incredible to the 
reader that by manipulating determinants in a particular way, we can solve linear sys- 
tems. 

In the next chapter we'll see another use for the determinant. Meanwhile, try to 
develop a deeper appreciation of math: odd, complicated things that seem completely 


22 version of Cramer’s Rule is often taught in introductory differential equations courses as it can be 
used to find solutions to certain linear differential equations. In this situation, the entries of the matrices 
are functions, not numbers, and hence computing determinants is easier than using Gaussian elimination. 
Again, though, as the matrices get large, other solution methods are resorted to. 
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unrelated often are intricately tied together. Mathematicians see these connections 
and describe them as “beautiful.” 


Exercises 3.5 


In Exercises 1-— 12, matrices A and Bare given. - —28 
(a) Give det(A) and det(A;) for all i. b= = 
(b) Use Cramer’s Rule to solve Ax = b. If 
Cramer’s Rule cannot be used to find 4 -4 0 16 
the solution, then state whether or not 9 A=/5 1 —1], b=|22 
a solution exists. 3 —1 2 8 
[7 =7 > 28 
WAS 7 o | "| 56 1 0 —10 
r 10. A= 4 —3 -10], 
9 5 > | —45 -9 6 —2 
maa 8, 5] a [G0] 
—40 
_[-8 16 ~  [—48 b= | —94 
AiO el ah È 
inis : al b= | | 7 -4 25 
L = z 11. A= | —2 1 -7 |, 
5 9 —7 34 
R a-l? a a 
[-1 3 19 E 
[7 14 > [-1 b= | —3 
a e 
3 0 -3 24 —6 —7 —7 
7.A=1}5 4 4], b=]0 12. A= 5 4 1 |, 
5 5 -4 31 5 4 8 
4 9 3 58 
8 A=|-5 -—2 —-13], b= | —35 
—1 10 -13 —49 
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EIGENVALUES AND EIGENVECTORS 


We have often explored new ideas in matrix algebra by making connections to our 
previous algebraic experience. Adding two numbers, x + y, led us to adding vectors 
X + y and adding matrices A + B. We explored multiplication, which then led us to 
solving the matrix equation AX = b, which was reminiscent of solving the algebra 
equation ax = b. 

This chapter is motivated by another analogy. Consider: when we multiply an un- 
known number x by another number such as 5, what do we know about the result? 
Unless, x = 0, we know that in some sense 5x will be “5 times bigger than x.” Applying 
this to vectors, we would readily agree that 5X gives a vector that is “5 times bigger 
than x.” Each entry in x is multiplied by 5. 

Within the matrix algebra context, though, we have two types of multiplication: 
scalar and matrix multiplication. What happens to x when we multiply it by a matrix 
A? Our first response is likely along the lines of “You just get another vector. There is 
no definable relationship.” We might wonder if there is ever the case where a matrix 
— vector multiplication is very similar to a scalar — vector multiplication. That is, do we 
ever have the case where Ax = ax, where a is some scalar? That is the motivating 
question of this chapter. 


4.1 Eigenvalues and Eigenvectors 


AS YOU READ 


1. T/F: Given any matrix A, we can always find a vector X where Ax = x. 
2. When is the zero vector an eigenvector for a matrix? 
3. If Vis an eigenvector of a matrix A with eigenvalue of 2, then what is AV? 


4. T/F: If Aisa5 x5 matrix, to find the eigenvalues of A, we would need to find the 
roots of a 5" degree polynomial. 
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We start by considering the matrix A and vector X as given below.? 


“Ey f 


Multiplying Ax gives: 


Wow! It looks like multiplying Ax is the same as 5x! This makes us wonder lots 
of things: Is this the only case in the world where something like this happens?2 Is A 
somehow a special matrix, and Ax = 5x for any vector X we pick?? Or maybe xX was a 
special vector, and no matter what 2 x 2 matrix A we picked, we would have AX = 5x.4 

A more likely explanation is this: given the matrix A, the number 5 and the vector 
X formed a special pair that happened to work together in a nice way. It is then natural 
to wonder if other “special” pairs exist. For instance, could we find a vector X where 
AX = 3X? 

This equation is hard to solve at first; we are not used to matrix equations where 
X appears on both sides of “=.” Therefore we put off solving this for just a moment to 
state a definition and make a few comments. 


Definition 31 Eigenvalues and Eigenvectors 
Let A be ann x n matrix, X a nonzero n x 1 column vector 


and À a scalar. If 
AX = XX, 


then x is an eigenvector of A and À is an eigenvalue of A. 


The word “eigen” is German for “proper” or “characteristic.” Therefore, an eigenvector 
of A is a “characteristic vector of A.” This vector tells us something about A. 

Why do we use the Greek letter À (lambda)? It is pure tradition. Above, we used a 
to represent the unknown scalar, since we are used to that notation. We now switch 
to À because that is how everyone else does it.? Don’t get hung up on this; À is just a 
number. 


1Recall this matrix and vector were used in Example 40 on page 79. 
2 Probably not. 

3 Probably not. 

4See footnote 2. 

5An example of mathematical peer pressure. 


4.1 Eigenvalues and Eigenvectors 


Note that our definition requires that A be a square matrix. If A isn’t square then 
Ax and AX will have different sizes, and so they cannot be equal. Also note that Xx must 
be nonzero. Why? What if X = 0? Then no matter what À is, AX = AX. This would 
then imply that every number is an eigenvalue; if every number is an eigenvalue, then 
we wouldn’t need a definition for it.Therefore we specify that x # 0. 

Our last comment before trying to find eigenvalues and eigenvectors for given ma- 
trices deals with “why we care.” Did we stumble upon a mathematical curiosity, or 
does this somehow help us build better bridges, heal the sick, send astronauts into 
orbit, design optical equipment, and understand quantum mechanics? The answer, of 
course, is “Yes”? This is a wonderful topic in and of itself: we need no external applica- 
tion to appreciate its worth. At the same time, it has many, many applications to “the 
real world.” Asimple Internet seach on “applications of eigenvalues” with confirm this. 

Back to our math. Given a square matrix A, we want to find a nonzero vector X 
and a scalar À such that AX = Ax. We will solve this using the skills we developed in 
Chapter 2. 


AX AX original equation 
AX— AX = 0 subtract AX from both sides 
(A = AIX O factor out X 


Think about this last factorization. We are likely tempted to say 
AX — AX = (A — A)x, 


but this really doesn’t make sense. After all, what does “a matrix minus a number” 
mean? We need the identity matrix in order for this to be logical. 

Let us now think about the equation (A — \/)X = 0. While it looks complicated, it 
really is just a matrix equation of the type we solved in Section 2.4. We are just trying 
to solve Bx = 0, where B = (A — AI). We know from our previous work that this 
type of equation’ always has a solution, namely, x = 0. However, we want x to be an 
eigenvector and, by the definition, eigenvectors cannot be 0. 

This means that we want solutions to (A — \/)x = 6 other than x = O. Recall 
that Theorem 8 says that if the matrix (A — A/) is invertible, then the only solution to 
(A—AI)X = Õis X = O. Therefore, in order to have other solutions, we need (A—X/) to 
not be invertible. Finally, recall from Theorem 17 that noninvertible matrices all have 
a determinant of 0. Therefore, if we want to find eigenvalues \ and eigenvectors x, we 
need det (A — AI) = 0. 

Let’s start our practice of this theory by finding \ such that det (A — A/) = 0; that 
is, let’s find the eigenvalues of a matrix. 


Example 95 Find the eigenvalues of A, thatis, find \ such that det (A — A/) = 0, 
where 


SExcept for the “understand quantum mechanics” part. Nobody truly understands that stuff; they just 
probably understand it. 
7Recall this is a homogeneous system of equations. 


179 


Chapter 4 Eigenvalues and Eigenvectors 


SOLUTION (Note that this is the matrix we used at the beginning of this sec- 
tion.) First, we write out what A — AI is: 


Therefore, 

det (A — AI) = 2 3—1) 
= (1— A)(3— A) -8 
=)?-4\-5 


es 4 | 


Since we want det (A — A) = 0, we want à? — 4\ — 5 = O. This is a simple 
quadratic equation that is easy to factor: 


-—4\-5=0 
(A—5)(A+1) =0 
A=-1,5 


According to our above work, det (A — A/) = O when à = —1, 5. Thus, the eigen- 
values of A are —1 and 5. 


Earlier, when looking at the same matrix as used in our example, we wondered if 
we could find a vector X such that AX = 3x. According to this example, the answer is 
“No.” With this matrix A, the only values of À that work are —1 and 5. 

Let’s restate the above in a different way: It is pointless to try to find x where 
Ax = 3x, for there is no such x. There are only 2 equations of this form that have a 
solution, namely 


> 


AX = —x and AX = 5X. 


As we introduced this section, we gave a vector X such that AX = 5x. Is this the 
only one? Let’s find out while calling our work an example; this will amount to finding 
the eigenvectors of A that correspond to the eigenvector of 5. 


Example 96 Find X such that AX = 5x, where 
1 4 
T 
SOLUTION Recall that our algebra from before showed that if 


AX =x then (A—AI)x =O. 
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Therefore, we need to solve the equation (A — AI)X = 0 for X when \ = 5. 


To solve (A—5/)x = 0, we form the augmented matrix and put it into reduced row 


echelon form: 
-4 4 0 wat 1 -1 0 
-2 0 mre o o ol 


Thus 


Xı =X 


X2 is free 


e- [z] [i] 


We have infinite solutions to the equation AX = 5X; any nonzero scalar multiple of the 


and 


1]. : : ‘ 
vector | | is a solution. We can do a few examples to confirm this: 


[> 3] [2}= [o] =5[2) 
B $) [23] - [28] =5/23]. 


Our method of finding the eigenvalues of a matrix A boils down to determining 
which values of A give the matrix (A—A/) a determinant of 0. In computing det (A — AI), 


we get a polynomial in \ whose roots are the eigenvalues of A. This polynomial is im- 
portant and so it gets its own name. 


Definition 32 Characteristic Polynomial 


Let A be ann x n matrix. The characteristic polynomial of A 
is the nt” degree polynomial p(\) = det (A — XI). 
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Our definition just states what the characteristic polynomial is. We know from our 
work so far why we care: the roots of the characteristic polynomial of an n x n matrix 
A are the eigenvalues of A. 

In Examples 95 and 96, we found eigenvalues and eigenvectors, respectively, of 
a given matrix. That is, given a matrix A, we found values À and vectors X such that 
Ax = AX. The steps that follow outline the general procedure for finding eigenvalues 
and eigenvectors; we’ll follow this up with some examples. 


Key Idea 14 Finding Eigenvalues and Eigenvectors 


Let A be ann x n matrix. 


1. To find the eigenvalues of A, compute p(X), the char- 


acteristic polynomial of A, set it equal to 0, then solve 
for À. 


2. To find the eigenvectors of A, for each eigenvalue 
solve the homogeneous system (A — AI)x = 0. 


Example 97 Find the eigenvalues of A, and for each eigenvalue, find an eigen- 
vector where 

—3 15 

sale), 
SOLUTION To find the eigenvalues, we must compute det (A — AI) and set it 

equal to 0. 

—3- xX 15 

det(A = A) =| 3 N 


Eg A)(9— A) -45 
=? —6\— 27—45 
=A*—6\ —72 

= (A — 12)(A + 6) 


Therefore, det (A — AI) = 0 when A = —6 and 12; these are our eigenvalues. (We 
should note that p(A) = A? — 6\ — 72 is our characteristic polynomial.) It sometimes 


helps to give them “names,” so we'll say 4; = —6 and à, = 12. Now we find eigen- 
vectors. 
For A; = —6: 


We need to solve the equation (A — (—6)/)x = 0. To do this, we form the appro- 
priate augmented matrix and put it into reduced row echelon form. 
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Our solution is 


Xı = —5xX2 


Xa is free; 
in vector form, we have 
x =x -3 
=| 4 |- 


We may pick any nonzero value for x2 to get an eigenvector; a simple option is x = 1. 
Thus we have the eigenvector 
gal 
= | a ie 


(We used the notation x; to associate this eigenvector with the eigenvalue A14.) 


We now repeat this process to find an eigenvector for Az = 12: 
In solving (A — 12/)x = 0, we find 


es 15 ol — E =] d 
rref 3 


3 —3 0 
X=x 1 
=x |l 


Again, we may pick any nonzero value for x2, and so we choose x2 = 1. Thus an 
eigenvector for Az is 


In vector form, we have 


To summarize, we have: 
: i . > -5 
eigenvalue A; = —6 with eigenvector x; = 1 


and 


= 1 
eigenvalue A2 = 12 with eigenvector x2 = H ; 
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We should take a moment and check our work: is it true that Ax; = A1Xx7? 
> —3 15 —5 


(=) 


— 1X4. 
Yes; it appears we have truly found an eigenvalue/eigenvector pair for the matrix A. 


Let’s do another example. 


-3 0 
Example 98 LetA = | 5 | . Find the eigenvalues of A and an eigenvector 


for each eigenvalue. 


SOLUTION We first compute the characteristic polynomial, set it equal to 0, 
then solve for À. 


—3-X 0 
5 1-2 
(—3 — A)(1 — A) 


det (A — AI) = | 


From this, we see that det (A — AI) = O when à = —3,1. We'll set \; = —3 and 
A2 = 1. 


Finding an eigenvector for A: 
We solve (A — (—3)/)x = 0 for X by row reducing the appropriate matrix: 


oss] a pe 


540 Et o 0 o 


Our solution, in vector form, is 


Again, we can pick any nonzero value for x2; a nice choice would eliminate the fraction. 
Therefore we pick x2 = 4, and find 
X = ma 
15| 4d: 


Finding an eigenvector for Az: 
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We solve (A — (1)/)x = 0 for X by row reducing the appropriate matrix: 


—4 0 0 ret 1 0 0 
5 0 0 0 0 Of” 


We’ve seen a matrix like this before,® but we may need a bit of a refreshing. Our 
first row tells us that x; = 0, and we see that no rows/equations involve x2. We con- 
clude that x2 is free. Therefore, our solution, in vector form, is 


aa 
X =X? 1l: 


We pick x, = 1, and find 


To summarize, we have: 
: : : 4 -5 
eigenvalue \; = —3 with eigenvector x; = 


and 


5 0 
eigenvalue \2 = 1 with eigenvector x) = | : 


So far, our examples have involved 2 x 2 matrices. Let’s do an example with a 3 x 3 
matrix. 


Example 99 Find the eigenvalues of A, and for each eigenvalue, give one eigen- 
vector, where 
—7 —2 10 
A=j|-3 2 3 
—6 -—2 9 
SOLUTION We first compute the characteristic polynomial, set it equal to 0, 


then solve for A. A warning: this process is rather long. We’ll use cofactor expansion 
along the first row; don’t get bogged down with the arithmetic that comes from each 
step; just try to get the basic idea of what was done from step to step. 


8See page 34. Our future need of knowing how to handle this situation is foretold in footnote 4. 
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lI 

l 
w 
N 
l 
- 
w 


det (A — ÀI) 


oy a8 | ~ -2| ee | + 10] -6 -2 
= (—7 — A)(à? — 11A + 24) + 2(34 — 9) + 10(—6) + 18) 
==)? $4)? — à = 6 

= —(A+1)(A— 2)(A — 3) 


(-7-A) 


In the last step we factored the characteristic polynomial — A? +47 — \—6. Factoring 
polynomials of degree > 2 is not trivial; we’ll assume the reader has access to methods 
for doing this accurately.’ 

Our eigenvalues are A; = —1, A2 = 2 and \3 = 3. We now find corresponding 
eigenvectors. 


For A, = —1: 


We need to solve the equation (A — (—1)!)x = 0. To do this, we form the appro- 
priate augmented matrix and put it into reduced row echelon form. 


—6 -—2 10 0 1 0 -1.5 0 
—3 3 3 0 rref 01 -5 0 
—6 -—2 10 0 0 0 0 0 


Our solution, in vector form, is 


X = x3 1/2 
1 


We can pick any nonzero value for x3; a nice choice would get rid of the fractions. 
3 


So we’ll set x3 = 2 and choose x; = | 1 | as our eigenvector. 
2 


For Az = 2: 


We need to solve the equation (A — 2/)x = 0. To do this, we form the appropriate 
augmented matrix and put it into reduced row echelon form. 


°You probably learned how to do this in an algebra course. As a reminder, possible roots can be found 
by factoring the constant term (in this case, —6) of the polynomial. That is, the roots of this equation could 
be +1, +2, +3 and +6. That’s 12 things to check. 
One could also graph this polynomial to find the roots. Graphing will show us that A = 3 looks like a root, 
and a simple calculation will confirm that it is. 
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—9 —2 10 0 24 10 -1 0 
—3 0 3 0 rref 0O 1 —5 0 
—6 —2 7 0 00 0 0 


Our solution, in vector form, is 


We can pick any nonzero value for x3; again, a nice choice would get rid of the frac- 
2 


tions. So we'll set x3 = 2 and choose x} = | 1 | as our eigenvector. 
2 


For A3 = 3: 


We need to solve the equation (A — 3/)x = 0. To do this, we form the appropriate 
augmented matrix and put it into reduced row echelon form. 


—10 —2 10 0 1 0 -1 0 
— 
—3 —1 3 0 ref |O 1 0 0 
—6 —2 6 0 00 0 0 
Our solution, in vector form, is (note that x, = 0): 
1 
x= X3 0 
1 
1 
We can pick any nonzero value for x3; an easy choice is x3 = 1, so x3 = | O | as 
1 


our eigenvector. 


To summarize, we have the following eigenvalue/eigenvector pairs: 


3 
eigenvalue \, = —1 with eigenvector x; = | 1 
2 


| 
| 


eigenvalue À = 2 with eigenvector x7 = 


eigenvalue \3 = 3 with eigenvector x3 = 


e Ore Ne N 


Let’s practice once more. 
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Example 100 Find the eigenvalues of A, and for each eigenvalue, give one eigen- 
vector, where 
2 -1 1 
A=j;0 1 6 
0 3 4 
SOLUTION We first compute the characteristic polynomial, set it equal to 0, 
then solve for A. We’ll use cofactor expansion down the first column (since it has lots 


of zeros). 


det (A — A) 


II 
oOo 
e 
l 
> 
fo) 


lI 
— 
N 

| 
> 
aie 


= (2—A)(A? — 5A — 14) 
= (2— A)(A—7)(A +2) 


Notice that while the characteristic polynomial is cubic, we never actually saw a 
cubic; we never distributed the (2 — A) across the quadratic. Instead, we realized that 
this was a factor of the cubic, and just factored the remaining quadratic. (This makes 
this example quite a bit simpler than the previous example.) 

Our eigenvalues are A; = —2, Ap = 2 and à; = 7. We now find corresponding 
eigenvectors. 


For Ay = —2: 


We need to solve the equation (A — (—2)!)x = 0. To do this, we form the appro- 
priate augmented matrix and put it into reduced row echelon form. 


4-110 1 0 3/4 0 
— 
0 3 60 rref O0 11 2 0 
0 3 60 00 0 0 
Our solution, in vector form, is 
—3/4 
X = X3 —2 
1 
We can pick any nonzero value for x3; a nice choice would get rid of the fractions. 
—3 
So we’ll set x3 = 4 and choose x; = | —8 | as our eigenvector. 
4 
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For Az = 2: 


We need to solve the equation (A — 2/)x = 0. To do this, we form the appropriate 
augmented matrix and put it into reduced row echelon form. 


0 -1 1 0 ay 0100 
0 -1 6 0 rref 0010 
0 3 20 000 0 


This looks funny, so we’ll look remind ourselves how to solve this. The first two 
rows tell us that x2 = 0 and x3 = O, respectively. Notice that no row/equation uses x1; 
we conclude that it is free. Therefore, our solution in vector form is 


1 
X=xX, 0 
0 


We can pick any nonzero value for x4; an easy choice is x; = 1 and choose x; = 
1 
O | as our eigenvector. 
0 


For \3 = 7: 


We need to solve the equation (A — 7/)x = 0. To do this, we form the appropriate 
augmented matrix and put it into reduced row echelon form. 


—5 -1 1 0 = 1 0 
oOo -6 6 0 rref 0O 1 -1 0 
0 3 -3 0 0 0 


Our solution, in vector form, is (note that x; = 0): 


0 
X= X3 1 
1 
0 
We can pick any nonzero value for x3; an easy choice is x3 = 1, so x3 = | 1] as 
1 


our eigenvector. 


To summarize, we have the following eigenvalue/eigenvector pairs: 


-3 
eigenvalue \; = —2 with eigenvector x; = | —8 
4 
1 
eigenvalue à = 2 with eigenvector x = | O 
(0) 
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(0) 
eigenvalue \3 = 7 with eigenvector x3 = | 1 
1 


In this section we have learned about a new concept: given a matrix A we can find 
certain values \ and vectors X where AX = AX. In the next section we will continue to 
the pattern we have established in this text: after learning a new concept, we see how 
it interacts with other concepts we know about. That is, we’ll look for connections 
between eigenvalues and eigenvectors and things like the inverse, determinants, the 
trace, the transpose, etc. 


Exercises 4.1 


In Exercises 1-6, a matrix A and one of its —1 
eigenvectors are given. Find the eigenvalue x=| 1 
of A for the given eigenvector. 1 
I 9 3 In Exercises 7 -— 11, a matrix A and one of its 
1. A= 6 s eigenvalues are given. Find an eigenvector of 
i A for the given eigenvalue. 
—4 
x= 16 6 
re | "E 
Zaca =$ | a 
|48 —15 Ac [z 6 | 
F —9 13 
x= J N=7 
L 3 E 
E —16 —28 —19 
saali A sala 69 46 
|-2 4 —42 —72 —49 
a2 A=5 
a il 7 -5 =10 
i 10. 4=|6 2 —6 
—11 —19 14 2 5 -5 
4. A=| -6 -8 6 \=-3 
—12 —22 15 
4 5 -3 
ee 11.A4=|-7 -8 3 
x= |2 1 -5 8 
4 \=2 
—7 1 3 In Exercises 12 — 28, find the eigenvalues of 
5. A= 10 2 —3 the given matrix. For each eigenvalue, give an 
—20 -14 1 eigenvector. 
1 —1 -4 
ze [> 12. | -3 = 
4 [-4 72 
13. 4 a] 
—12 -10 0 L 
z5 15 13 0 i [2 =| 
15 18 -5 ' |2 —8 


15. 


16. 


17. 


18. 


19. 


20. 


21. 


22. 
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[3 12 1 0 —18 
|1 -1 23. |-4 3 -1 
rs g 1 0 -8 
Lot 8 -1 18 0 
[3 =i 24. /1 2 0 

1 3 5 -3 -1 

0 1 5 0 0 
|25 0 25. |1 1 0 

-3 1 =1 5 -2 

0 -1 
z 2 -1 1 

1 -2 -3 26. /0 3 6 

0 3 0 0 0 7 

0 -1 -1 

Se aoe a 3 5 -5 

ok aie U 27. |-2 3 2 

@ -1 3 =e 0 

1 0 12 121 

2 -5 0 28. |1 2 3 

1 0 2 1 1 1 


Properties of Eigenvalues and Eigenvectors 


AS YOU READ 


1. 


2, 


T/F: A and A’ have the same eigenvectors. 
T/F: A and AT! have the same eigenvalues. 
T/F: Matrices with a trace of 0 are important, although we haven’t seen why. 


T/F: A matrix A is invertible only if 1 is an eigenvalue of A. 


In this section we’ll explore how the eigenvalues and eigenvectors of a matrix relate 
to other properties of that matrix. This section is essentially a hodgepodge of inter- 
esting facts about eigenvalues; the goal here is not to memorize various facts about 
matrix algebra, but to again be amazed at the many connections between mathemat- 
ical concepts. 

We'll begin our investigations with an example that will give a foundation for other 
discoveries. 


Example 101 Let A = 


OOF 


2 3 
4 5 |. Find the eigenvalues of A. 
0 6 
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SOLUTION To find the eigenvalues, we compute det (A — AI): 
1-> 2 3 
det(A-\)=| 0 4-A 5 
0 0 6—A 


= (1— A)(4— A)(6— A) 


Since our matrix is triangular, the determinant is easy to compute; it is just the 
product of the diagonal elements. Therefore, we found (and factored) our character- 
istic polynomial very easily, and we see that we have eigenvalues of À = 1, 4, and 6. 


This examples demonstrates a wonderful fact for us: the eigenvalues of a triangular 
matrix are simply the entries on the diagonal. Finding the corresponding eigenvectors 


still takes some work, but finding the eigenvalues is easy. 


With that fact in the backs of our minds, let us proceed to the next example where 
we will come across some more interesting facts about eigenvalues and eigenvectors. 


3 15 —7 —2 10 
Example 102 LetA = and let B = |-3 2 3 
2 2 -6 —2 9 


Examples 97 and 99, respectively). Find the following: 
1. eigenvalues and eigenvectors of A and B 
2. eigenvalues and eigenvectors of A~+ and B71 
3. eigenvalues and eigenvectors of A’ and B7 
4. The trace of A and B 


5. The determinant of A and B 


SOLUTION We’ll answer each in turn. 


(as used in 


1. We already know the answer to these for we did this work in previous examples. 


Therefore we just list the answers. 


For A, we have eigenvalues \ = —6 and 12, with eigenvectors 


X =X) | 1 | and x H , respectively. 


For B, we have eigenvalues \ = —1, 2, and 3 with eigenvectors 
3 2 1 
xX =x3]1],x3]11] and x |O | , respectively. 
2 2 1 
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2. We first compute the inverses of A and B. They are: 


-4 1/3 13/3 
_, [1/8 5/24 7 
‘i ala A ee E = a n 


Finding the eigenvalues and eigenvectors of these matrices is not terribly hard, 
but it is not “easy,” either. Therefore, we omit showing the intermediate steps 
and go right to the conclusions. 


For A~+, we have eigenvalues \ = —1/6 and 1/12, with eigenvectors 


= 1 : 
X =X) 1 and x2 117 respectively. 


For B-+, we have eigenvalues \ = —1, 1/2 and 1/3 with eigenvectors 
3 2 1 
X=x3]1],x3]1] and x |O | , respectively. 
2 2 1 


3. Of course, computing the transpose of A and B is easy; computing their eigenval- 
ues and eigenvectors takes more work. Again, we omit the intermediate steps. 


For A’, we have eigenvalues À = —6 and 12 with eigenvectors 


= 5 : 
X =X) 1 and x2 11? respectively. 


For B’, we have eigenvalues \ = —1, 2 and 3 with eigenvectors 
—1 —1 0 
X=x3| 0 | ,x3}] 1 and x3 | —2 | , respectively. 
1 1 1 


4. The trace of A is 6; the trace of B is 4. 


5. The determinant of A is —72; the determinant of B is —6. 


Now that we have completed the “grunt work,” let’s analyze the results of the pre- 
vious example. We are looking for any patterns or relationships that we can find. 


The eigenvalues and eigenvectors of A and A~?. 


In our example, we found that the eigenvalues of A are —6 and 12; the eigenvalues 
of A~+ are —1/6 and 1/12. Also, the eigenvalues of B are —1, 2 and 3, whereas the 
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eigenvalues of B™t are —1, 1/2 and 1/3. There is an obvious relationship here; it 
seems that if À is an eigenvalue of A, then 1/, will be an eigenvalue of A~*. We can 
also note that the corresponding eigenvectors matched, too. 

Why is this the case? Consider an invertible matrix A with eigenvalue \ and eigen- 
vector X. Then, by definition, we know that Ax = AX. Now multiply both sides by 
Aa}: 


AX = AX 
—1a>ż_— —1\ > 
A “AX = A AX 
X = MTX 
1, 5 
ae A 


We have just shown that A~?X = 1/AX; this, by definition, shows that x is an eigen- 
vector of A~! with eigenvalue 1/X. This explains the result we saw above. 


The eigenvalues and eigenvectors of A and A’. 


Our example showed that A and A’ had the same eigenvalues but different (but 
somehow similar) eigenvectors; it also showed that B and B” had the same eigenvalues 
but unrelated eigenvectors. Why is this? 

We can answer the eigenvalue question relatively easily; it follows from the prop- 
erties of the determinant and the transpose. Recall the following two facts: 


1. (A +B)" =A’ + B' (Theorem 12) and 
2. det (A) = det (A’) (Theorem 17). 


We find the eigenvalues of a matrix by computing the characteristic polynomial; 
that is, we find det (A — AI). What is the characteristic polynomial of A’? Consider: 


det (A7 — Al) = det (A7 — XI") since / = /" 
= det ((A — \/)’) Theorem 12 
= det (A — Al) Theorem 17 


So we see that the characteristic polynomial of A’ is the same as that for A. There- 
fore they have the same eigenvalues. 

What about their respective eigenvectors? Is there any relationship? The simple 
answer is “No.” 1° 


10We have defined an eigenvector to be a column vector. Some mathematicians prefer to use row vectors 
instead; in that case, the typical eigenvalue/eigenvector equation looks like XA = AX. It turns out that doing 
things this way will give you the same eigenvalues as our method. What is more, take the transpose of the 
above equation: you get (xA)’ = (Ax)! which is also ATX’ = Ax’. The transpose of a row vector is a 
column vector, so this equation is actually the kind we are used to, and we can say that x” is an eigenvector 
of A’. 

In short, what we find is that the eigenvectors of A’ are the “row” eigenvectors of A, and vice-versa. 


4.2 Properties of Eigenvalues and Eigenvectors 


The eigenvalues and eigenvectors of A and The Trace. 


Note that the eigenvalues of A are —6 and 12, and the trace is 6; the eigenvalues 
of B are —1, 2 and 3, and the trace of B is 4. Do we notice any relationship? 


It seems that the sum of the eigenvalues is the trace! Why is this the case? 


The answer to this is a bit out of the scope of this text; we can justify part of this 
fact, and another part we’ll just state as being true without justification. 


First, recall from Theorem 14 that tr(AB) =tr(BA). Secondly, we state without jus- 
tification that given a square matrix A, we can find a square matrix P such that P~*AP is 
an upper triangular matrix with the eigenvalues of A on the diagonal. Thus tr(P~*AP) 
is the sum of the eigenvalues; also, using our Theorem 14, we know that tr(P-+AP) = 
tr(P-1PA) =tr(A). Thus the trace of A is the sum of the eigenvalues. 


The eigenvalues and eigenvectors of A and The Determinant. 


Again, the eigenvalues of A are —6 and 12, and the determinant of A is —72. The 
eigenvalues of B are —1, 2 and 3; the determinant of B is —6. It seems as though the 
product of the eigenvalues is the determinant. 


This is indeed true; we defend this with our argument from above. We know that 
the determinant of a triangular matrix is the product of the diagonal elements. There- 
fore, given a matrix A, we can find P such that P~*AP is upper triangular with the 
eigenvalues of A on the diagonal. Thus det (P~1AP) is the product of the eigenvalues. 
Using Theorem 17, we know that det (P~*AP) = det (PPA) = det (A). Thus the 
determinant of A is the product of the eigenvalues. 


We summarize the results of our example with the following theorem. 


11Who in the world thinks up this stuff? The answer is Marie Ennemond Camille Jordan. 
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Theorem 20 Properties of Eigenvalues and Eigenvectors 


Let A be ann x n invertible matrix. The following are true: 


1. If Ais triangular, then the diagonal elements of A are 
the eigenvalues of A. 


. If À is an eigenvalue of A with eigenvector x, then 1 is 
an eigenvalue of A71 with eigenvector x. 


. If À is an eigenvalue of A then A is an eigenvalue of 
AS 


. The sum of the eigenvalues of A is equal to tr(A), the 
trace of A. 


. The product of the eigenvalues of A is the equal to 
det (A), the determinant of A. 


There is one more concept concerning eigenvalues and eigenvectors that we will 
explore. We do so in the context of an example. 


Example 103 Find the eigenvalues and eigenvectors of the matrix A = | i ; | ; 
SOLUTION To find the eigenvalues, we compute det (A — AI): 
1-A 2 
det (A = A) =| 1 A 
= (1— A)(2— A) -2 
=) —3) 
= A(A — 3) 


Our eigenvalues are therefore \ = 0, 3. 
For A = 0, we find the eigenvectors: 


1 2 0; Fe |1 2 0 
1 2 0 0 0 0 


This shows that x; = —2x>, and so our eigenvectors X are 


X=x “2 
etal 
For A = 3, we find the eigenvectors: 


196 


4.2 Properties of Eigenvalues and Eigenvectors 


-2 2 0] — [1 -1 0 
1 -1 0| "F Jo o o 


This shows that x; = x2, and so our eigenvectors X are 
xXx =x 1 
=x] l 


One interesting thing about the above example is that we see that 0 is an eigen- 
value of A; we have not officially encountered this before. Does this mean anything 
significant? 1? 

Think about what an eigenvalue of 0 means: there exists an nonzero vector x where 
AX = 0X = O. That is, we have a nontrivial solution to AX = 0. We know this only 
happens when A is not invertible. 

So if A is invertible, there is no nontrivial solution to AX = 0, and hence 0 is not 
an eigenvalue of A. If A is not invertible, then there is a nontrivial solution to Ax = 0, 
and hence 0 is an eigenvalue of A. This leads us to our final addition to the Invertible 
Matrix Theorem. 


Theorem 21 Invertible Matrix Theorem 


Let A be ann xn matrix. The following statements are equiv- 
alent. 


(a) A is invertible. 


(h) A does not have an eigenvalue of 0. 


This section is about the properties of eigenvalues and eigenvectors. Of course, we 
have not investigated all of the numerous properties of eigenvalues and eigenvectors; 
we have just surveyed some of the most common (and most important) concepts. 
Below are three quick examples of the many things that still exist to be explored, two 
of which will be considered in the next section. 

First, recall the matrix 

1 4 
“=| 3] 


that we used in Example 95. It’s characteristic polynomial is p(A) = X 4—5. 
Compute p(A); that is, compute A? — 4A — 5I. You should get something “interesting,” 


since 0 is a “special” number, we might think so — after all, we found that having a determinant of 0 is 
important. Then again, a matrix with a trace of 0 isn’t all that important. (Well, as far as we have seen; it 
actually is). So, having an eigenvalue of 0 may or may not be significant, but we would be doing well if we 
recognized the possibility of significance and decided to investigate further. 
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and you should wonder “does this always work?” 

Second, in all of our examples, we have considered matrices where eigenvalues 
“appeared only once.” Since we know that the eigenvalues of a triangular matrix ap- 
pear on the diagonal, we know that the eigenvalues of 


1 1 
A = 
are “1 and 1;” that is, the eigenvalue \ = 1 appears twice. What does that mean when 
we consider the eigenvectors of A = 1? Compare the result of this to the matrix 


1 0 
a= [o sh 


which also has the eigenvalue \ = 1 appearing twice.1 We’ll look at some conse- 
quences of this in the next section. 
Third, consider the matrix 
0 —1 
A= : 


What are the eigenvalues? We quickly compute the characteristic polynomial to be 
pA) = A? + 1. Therefore the eigenvalues are +,/—1 = +i—complex eigenvalues! 
We'll look at this in the next section as well. 

Finally, a disclaimer: we have found the eigenvalues of matrices by finding the roots 
of the characteristic polynomial. We have limited our examples to quadratic and cubic 
polynomials; one would expect for larger sized matrices that a computer would be 
used to factor the characteristic polynomials. However, in general, this is not how the 
eigenvalues are found. Factoring high order polynomials is too unreliable, even with 
a computer — round off errors can cause unpredictable results. Also, to even compute 
the characteristic polynomial, one needs to compute the determinant, which is also 
expensive (as discussed in the previous chapter). 

So how are eigenvalues found? There are iterative processes that can progressively 
transform a matrix A into another matrix that is almost an upper triangular matrix (the 
entries below the diagonal are almost zero) where the entries on the diagonal are the 
eigenvalues. The more iterations one performs, the better the approximation is. These 
methods are so fast and reliable that some computer programs convert polynomial 
root finding problems into eigenvalue problems! 

Most textbooks on Linear Algebra or Numerical Analysis will provide direction on 
exploring the above topics and give further insight to what is going on. We have men- 
tioned all the eigenvalue and eigenvector properties in this section for the same rea- 
sons we gave in the previous section. First, knowing these properties helps us solve 
numerous real world problems, and second, it is fascinating to see how rich and deep 
the theory of matrices is. 
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13yes, but this is for a later course in linear algebra. 

14To direct further study, it helps to know that mathematicians refer to this as the algebraic multiplicity 
of an eigenvalue. In each of these two examples, A has the eigenvalue à = 1 with algebraic multiplicity of 
2. 

15Be careful; this matrix is not triangular. 
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Exercises 4.2 


In Exercises 1- 6, a matrix A is given. For 2. [ -2 =| 
each, | —1 3 
(a) Find the eigenvalues of A, and for each 3 [5 30 
eigenvalue, find an eigenvector. “|-1 —6 
(b) Do the same for A’. r T4 72 | 
(c) Do the same for A~?. [71 13 
(d) Find tr(A). 5 -9 0 
. 5 1 -5 0 
(e) Find det (A). 5 A 3 
Use Theorem 20 to verify your results. 
0 25 0 
1 O 4 6. |1 0 0 
“}-1 5 1 1 -3 


4.3 Matrices With Complex and Defective Eigenvalues 


AS YOU READ 


1. T/F: An invertible matrix must have real eigenvalues. 

2. T/F: If A has real entries and A is an eigenvalue for A, so is À. 

3. T/F: Ann x n matrix can have n + 1 distinct eigenvalues. 

4. T/F: If A and B are matrices with complex entries then Attb = A ttb. 


5. T/F: All the matrix algebra we’ve done works if the matrix has complex entries. 


This section focuses primarily on matrices with complex eigenvalues and eigenvec- 
tors, though we start with a simple example showing that even if A is a matrix with real 
eigenvalues, some subtleties can occur. 


Example 104 LetA = f il . Find the eigenvalues and eigenvectors of A. 
SOLUTION To find the eigenvalues, we solve det (A — AI) = 0: 
1-AX 1 
det (A = A) =| 0 aN 
= (1-A) 
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This matrix has “two” eigenvalues, both of which happen to equal 1. To find the cor- 
responding eigenvector(s) we consider the system (A — /)x = 0, or 


O 1 Xi} 0 

o Oj] |x| lol? 
A straightforward back substitution shows that x. = O and x; is free. That is, the 
eigenvectors for A with eigenvalue 1 are of the form 


= 1 
Xı = X1 A 


for any nonzero choice of x,. All eigenvectors for eigenvalue 1 are multiples of a single 
vector. 


Contrast the matrix in Example 104 with the 2 x 2 identity matrix, which also has 
“two copies” of the eigenvalue1, but for which all vectors in two-dimensional space 
are eigenvectors. In each case À = 1 is a double root of the characteristic polynomial 
p(\) = (1—)?, and hence \ = 1is an eigenvalue of “algebraic multiplicity” 2. In the 
cases we’ve seen so far, for a2 x 2 matrix we can find two “independent” eigenvectors 
(not multiples of each other), but this is not the case for the matrix in Example 104. 
For that matrix the eigenvalue 1 is said to be defective. 

For larger matrices the situation can be even more complicated, but that is a topic 
left to a more advanced linear algebra course. We will encounter defective eigenvalues 
in the 2 x 2 case when we start analyzing systems of differential equations, and we'll 
deal with that specific situation at that time. 

To continue our exploration of eigenvalues and eigenvectors, let us consider a ma- 
trix introduced at the end of the last section. As it turns out, since the eigenvalues are 
roots of a polynomial, they can be complex! 


0 -1 . i : 
Example 105 LetA = Ei 0 | . Find the eigenvalues and eigenvectors of A. 
SOLUTION Before proceeding with the computation it’s worth noting that this 


matrix implements a 90 degree counterclockwise rotation in the plane, for if x is any 
vector then the vector y defined by 


is orthogonal to X and has the same length as xX. To see this note that x - y = 0 and 
||x|| = |ly||. It’s also easy to check that multiplication by A maps first quadrant vectors 
to the second quadrant, second quadrant to the third, and so on. An eigenvector x 
for A is supposed to remain parallel under multiplication by A (AX = AX) and yet this 
matrix rotates vectors by 90 degrees. How can a nonzero vector both remain parallel 
to its original orientation and be rotated at the same time!? 
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Let’s proceed with the computation and see what happens. We need the roots of 
the characteristic equation, computed as 


O-A -1 
det(A = A) =| 1 EN 
=) +1 
The eigenvalues are A = +i, both complex. This is perfectly acceptable and indeed, 


“most” matrices of any size have complex eigenvalues. Let’s proceed with the compu- 
tation of an eigenvector for each eigenvalue. 
Let’s do A = i first. We need to solve (A — il)x = 0, leading to 


—j —1 Xi} 0 

1 —i X2 E 0} 
Here’s an important fact: all of the matrix algebra we’ve done to this point—linear sys- 
tems, Gaussian elimination, matrix algebra, etc.—works just fine with complex num- 
bers, for the simple reason that all of material so far relied on our ability to add, sub- 


tract, multiply, and divide real numbers. We know how to do all of this for complex 
numbers too! So we form the augmented matrix 


—i -1 0 
1 -i 0 


and perform a Gauss-Jordan elimination. The row operation Ra — 1/(—/)R, (divide 
row 1 by —i, or equivalently, multiply by i) results in 


and then (—1)Rı + R2 —> R> yields 


1 —i 0 
o O O|' 


This is in reduced row echelon form. We see that x2 is a free variable and the top row 
embodies the equation x; — ix = 0, so xı = ix2. That is, anything of the form 


~ i 
z=% |i] 


with x, Æ 0 is an eigenvector with eigenvalue i. The eigenvector itself here has com- 
plex components! 
A similar computation for A = —i leads to (A + il)X = 0 with augmented matrix 


i —1 0 
1 i 0f? 
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The operations R; — —iR, followed by Rı + R2 —> R> yield 
1 i 0 
o 0 O|° 


with solutions in which x3 is free and 


which is an eigenvector with eigenvalue —i for any nonzero choice of x2. 


The punchline is this: a matrix can have complex eigenvalues and the associated 
eigenvectors are typically complex. Let’s do another example. 


Example 106 LetA = | k | . Find the eigenvalues and eigenvectors of A. 
SOLUTION We begin by finding the characteristic polynomial, 
2-2 3 
det (A =A) = | 3 A 
=) — 4) +13 


The eigenvalues are (via the quadratic formula) A = 2 + 3i and À = 2 — 3i. Note that 
since the characteristic polynomial has real coefficients (since A has real coefficients) 
the roots are conjugate to each other, that is, the eigenvalues are conjugate. 

We proceed with the computation of an eigenvector for each eigenvalue. Let’s do 
À = 2 + 3i first. We need to solve (A — (2 + 3i) NX = O, leading to 


[3 alilo] 


-3i 3 0 
—3 —3i 0|“ 


The row operations R4 — (i/3)R followed by —3R; + R2 —> R> yields 
1 i O 
0 0 O|° 


Here x is free. Back substitute to find that anything of the form 


m -i 


with x2 Æ 0 is an eigenvector with eigenvalue 3 + 2i. 


The augmented matrix is 
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A similar computation for \ = 3 — 2i yields eigenvectors of the form 
x=x,|! 
=%/ 4 
Note that the eigenvectors are themselves conjugate to each other, component by 


component. 


Let’s do one more slightly larger example. 


8 -5 0 
Example 107 LetA=|4 0 0 |. Find the eigenvalues and eigenvectors 
6 —12 -2 
of A. 
SOLUTION The characteristic polynomial is a cubic, given by 
8-—A —5 (0) 
det (A — Àl) = 4 —À 0 
6 —12 -2-AX 
= X? — 6)? +424 40 
= 0. 


The characteristic polynomial factors as (A + 2)(A — (4 + 2/))(A — (4 — 2/)), so the 
eigenvalues of A are A = —2,4 + 2i, and 4 — 2i. 
The eigenvector for A = 4 + 2i is given by solving (A — (4 + 2/)I)x 


in augmented matrix 


l 


= 0 and results 


4- 2i —5 0 0 
4 —4—2i 0 0 
6 —12 —6—2i 0 


Gauss-Jordan elimination produces 
10 1/⁄3+i 0 
O 1 2/342i/3 0 
0 o0 (0) (0) 


Since the x3 column has no pivot, x3 is free. The remaining two equations yield x 
—(2/3 + 2i/3)x3 and x; = —(1/3 + /)x3. The eigenvector is any nonzero multiple 
—2/3 — 2/3 
—1/3 — i 
1 


Xi = X3 


A similar computation with À = 4 — 2i yields eigenvector 
—2/3 + 2i/3 
—1/3 +i 


X% = X3 
1 
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The eigenvector for \ = —2 is, via similar computations (with no complex numbers) 
0 
X% = X3 | 0 
1 


All in all the eigenvalues and eigenvectors for A are 


—2/3 — 2i/3 
eigenvalue Àq = 4 + 2i with eigenvector x; = —1/3-i 
1 
—2/3 + 21/3 
eigenvalue à = 4 — 2i with eigenvector x7 = —1/3 +i 
1 
(0) 
eigenvalue \3 = —2 with eigenvector x3 = | 0 
1 


or scalar multiples thereof. 


The Eigenvalues and Eigenvectors of Real Matrices. 


In Examples 105, 106, and 107, it’s hard not to notice that the eigenvalues come 
in conjugate pairs (or are real) and that the eigenvectors corresponding to complex- 
conjugate eigenvalues are also complex-conjugates of each other, component by com- 
ponent. This is no coincidence! 

To see why, first note that from Appendix section A.1, complex conjugation dis- 
tributes over all of the basic arithmetic operations, addition, subtraction, multiplica- 
tion, and division. Thus for example, if ï and V are two n-dimensional vectors with 
complex components we have 


UV = Uv + UgV2 UnVn 
= UV + U2V2 UnVn 
= Uy Vy + U2 V2 + + Un Vn 
=0-V 


where conjugation of a vector (or matrix) is done component-by-component. Thus the 
“conjugate of the dot product is the dot product of the conjugates.” Similar computa- 
tions show that in fact AB = A B for any matrices (or vectors) A and B. 


With the above facts in hand we can show 
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Theorem 22 Complex Eigenvalues/Eigenvectors of a Real Matrix 


Let A be ann x n matrix with real entries. If A is a complex 
eigenvalue for A with eigenvector x then A is also an eigen- 
value for A with eigenvector x. 


Proof: To say that x is an eigenvector for A with eigenvector A means that 
AX = AX. 


Both sides above are vectors; we conjugate each component-by-component and con- 
clude that Ax = AX. But by using the above discussed properties of conjugation we 
can distribute the conjugation over the products. This yields 


Ax=Xx. 
But since A is real, A = A and we have 
AX=Xx* 


This last equation is exactly the statement that X is an eigenvector for A with eigen- 
value À. 


Exercises 4.3 


For each matrix in 1-13 find the eigenvalues 2 1 0 
and all corresponding eigenvectors. 8 jO 2 1 
0 0 2 
1 2 0 
“10 2 
17 7 —4 
2. k J §, | -23 -9 8 
me 6 2 4 
1 -1 
3. 
k a] 5 -3 0 
7 5 10. 6 —4 0 
4. E i -13 7 4 
-9 -5 
5. | 8 3 1 -13 0 
11. 4 -7 0 
2 4 6 9 3 
6. ie $l 
2 1 0 6 —6 2 
7. |O0 2 0 12. -8 19 —14 
0 0 2 —15 30 —-20 
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GRAPHICAL EXPLORATIONS OF VECTORS 


We already looked at the basics of graphing vectors. In this chapter, we’ll explore these 
ideas more fully. One often gains a better understanding of a concept by “seeing” it. 
For instance, one can study the function f(x) = x? and describe many properties of 
how the output relates to the input without producing a graph, but the graph can 
quickly bring meaning and insight to equations and formulae. Not only that, but the 
study of graphs of functions is in itself a wonderful mathematical world, worthy of 
exploration. 

We’ve studied the graphing of vectors; in this chapter we’ll take this a step further 
and study some fantastic graphical properties of vectors and matrix arithmetic. We 
mentioned earlier that these concepts form the basis of computer graphics; in this 
chapter, we’ll see even better how that is true. 


5.1 Transformations of the Cartesian Plane 


AS YOU READ 


1. To understand how the Cartesian plane is affected by multiplication by a matrix, 
it helps to study how what is affected? 


2. Transforming the Cartesian plane through matrix multiplication transforms straight 
lines into what kind of lines? 


3. T/F: If one draws a picture of a sheep on the Cartesian plane, then transformed 
the plane using the matrix 
—1 0 
0o 1)’ 


one could say that the sheep was “sheared.” 
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We studied in Section 2.3 how to visualize vectors and how certain matrix arith- 
metic operations can be graphically represented. We limited our visual understanding 
of matrix multiplication to graphing a vector, multiplying it by a matrix, then graphing 
the resulting vector. In this section we’ll explore these multiplication ideas in greater 
depth. Instead of multiplying individual vectors by a matrix A, we’ll study what hap- 
pens when we multiply every vector in the Cartesian planes by A.* 


Because of the Distributive Property, demonstrated way back in Example 41, we 
can say that the Cartesian plane will be transformed in a very nice, predictable way. 
Straight lines will be transformed into other straight lines (and they won’t become 
curvy, or jagged, or broken). Curved lines will be transformed into other curved lines 
(perhaps the curve will become “straight,” but it won’t become jagged or broken). 

One way of studying how the whole Cartesian plane is affected by multiplication by 
a matrix A is to study how the unit square is affected. The unit square is the square with 
corners at the points (0,0), (1,0), (1,1), and (0,1). Each corner can be represented 
by the vector that points to it; multiply each of these vectors by A and we can get an 
idea of how A affects the whole Cartesian plane. 

Let’s try an example. 


Example 108 Plot the vectors of the unit square before and after they have been 
multiplied by A, where 


SOLUTION The four corners of the unit square can be represented by the vec- 


fh [ol lal {al 


Multiplying each by A gives the vectors 
0 1 5 4 
op’ 2? 5|? 3” 
respectively. 


(Hint: one way of using your calculator to do this for you quickly is to make a 2 x 4 
matrix whose columns are each of these vectors. In this case, create a matrix 


Then multiply B by A and read off the transformed vectors from the respective columns: 


0154 
a 2 = 3: 


1No, we won’t do them one by one. 
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This saves time, especially if you do a similar procedure for multiple matrices A. Of 
course, we can save more time by skipping the first column; since it is the column of 
zeros, it will stay the column of zeros after multiplication by A.) 

The unit square and its transformation are graphed in Figure 5.1, where the shaped 
vertices correspond to each other across the two graphs. Note how the square got 
turned into some sort of quadrilateral (it’s actually a parallelogram). A really interest- 
ing thing is how the triangular and square vertices seem to have changed places — it is 
as though the square, in addition to being stretched out of shape, was flipped. 


y y 


Figure 5.2: Emphasizing straight lines going to straight lines in Example 108. 


To stress how “straight lines get transformed to straight lines,” consider Figure 5.2. 
Here, the unit square has some additional points drawn on it which correspond to the 
shaded dots on the transformed parallelogram. Note how relative distances are also 
preserved; the dot halfway between the black and square dots is transformed to a po- 
sition along the line, halfway between the black and square dots. 


Much more can be said about this example. Before we delve into this, though, let’s 
try one more example. 
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Example 109 Plot the transformed unit square after it has been transformed by 
A, where 
0 -1 
A= : 
SOLUTION We'll put the vectors that correspond to each corner in a matrix B 


as before and then multiply it on the left by A. Doing so gives: 
O -1//0 1 1 0 
a ae a 
_{0 O —1 -1 
“10 1 1 (0) 
In Figure 5.3 the unit square is again drawn along with its transformation by A. 


y y 


Figure 5.3: Transforming the unit square by matrix multiplication in Example 109. 


Make note of how the square moved. It did not simply “slide” to the left;? nor 
did it “flip” across the y axis. Rather, it was rotated counterclockwise about the origin 
90°. In a rotation, the shape of an object does not change; in our example, the square 
remained a square of the same size. 


We have broached the topic of how the Cartesian plane can be transformed via 
multiplication by a2 x 2 matrix A. We have seen two examples so far, and our intuition 
as to how the plane is changed has been informed only by seeing how the unit square 
changes. Let’s explore this further by investigating two questions: 


1. Suppose we want to transform the Cartesian plane in a known way (for instance, 
we may want to rotate the plane counterclockwise 180°). How do we find the 
matrix (if one even exists) which performs this transformation? 


2. How does knowing how the unit square is transformed really help in understand- 
ing how the entire plane is transformed? 


These questions are closely related, and as we answer one, we will help answer 
the other. 


2mathematically, that is called a translation 
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To get started with the first question, look back at Examples 108 and 109 and con- 
sider again how the unit square was transformed. In particular, is there any correlation 
between where the vertices ended up and the matrix A? 

If you are just reading on, and haven’t actually gone back and looked at the exam- 
ples, go back now and try to make some sort of connection. Otherwise — you may have 
noted some of the following things: 


1. The zero vector (0, the “black” corner) never moved. That makes sense, though; 


Th 
AO = 0. 


2. The “square” corner, i.e., the corner corresponding to the vector f 


| 
, is always 


transformed to the vector in the first column of A! 


3. Likewise, the “triangular” corner, i.e., the corner corresponding to the vector 


oj. A; 
ac is always transformed to the vector in the second column of A!? 
4. The “white dot” corner is always transformed to the sum of the two column 


vectors of A.’ 


Let’s now take the time to understand these four points. The first point should be 
clear; Owill always be transformed to Ovia matrix multiplication. (Hence the hint in the 
middle of Example 108, where we are told that we can ignore entering in the column 
of zeros in the matrix B.) 

We can understand the second and third points simultaneously. Let 


a b ms ll + |0 
a=|? al: asig] and é=|t]. 


What are Ae; and Ae}? 


3 Although this is less of a surprise, given the result of the previous point. 
4This observation is a bit more obscure than the first three. It follows from the fact that this corner of 
the unit square is the “sum” of the other two nonzero corners. 
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So by mere mechanics of matrix multiplication, the square corner êi is transformed 
to the first column of A, and the triangular corner e3 is transformed to the second col- 
umn of A. A similar argument demonstrates why the white dot corner is transformed 
to the sum of the columns of A.” 


Revisit now the question “How do we find the matrix that performs a given trans- 
formation on the Cartesian plane?” The answer follows from what we just did. Think 
about the given transformation and how it would transform the corners of the unit 
square. Make the first column of A the vector where e; goes, and make the second 
column of A the vector where e} goes. 


Let’s practice this in the context of an example. 


Example 110 Find the matrix A that flips the Cartesian plane about the x axis and 
then stretches the plane horizontally by a factor of two. 


: : = 1 ‘ 

SOLUTION We first consider ej = lo . Where does this corner go to under 

the given transformation? Flipping the plane across the x axis does not change é; at 
. = 2 . _ | 2 
all; stretching the plane sends e; to B . Therefore, the first column of A is B : 
. S 0 Re : 5 
Now consider e3 = il Flipping the plane about the x axis sends e; to the vec- 
(0) . : F 

tor | 1 | ; subsequently stretching the plane horizontally does not affect this vector. 


Therefore the second column of A is | kA : 


Putting this together gives 


To help visualize this, consider Figure 5.4 where a shape is transformed under this 
matrix. Notice how it is turned upside down and is stretched horizontally by a factor 
of two. (The gridlines are given as a visual aid.) 


5 Another way of looking at all of this is to consider what A - I is: of course, it is just A. What are the 
columns of /? Just êj and &. 
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t ' 


Figure 5.4: Transforming the Cartesian plane in Example 110 


Awhile ago we asked two questions. The first was “How do we find the matrix that 
performs a given transformation?” We have just answered that question (although we 
will do more to explore it in the future). The second question was “How does knowing 
how the unit square is transformed really help us understand how the entire plane is 
transformed?” 

Consider Figure 5.5 where the unit square (with vertices marked with shapes as 
before) is shown transformed under an unknown matrix. How does this help us un- 
derstand how the whole Cartesian plane is transformed? For instance, how can we 
use this picture to figure out how the point (2, 3) will be transformed? 


y 


A 


Figure 5.5: The unit square under an unknown transformation. 


There are two ways to consider the solution to this question. First, we know now 
how to compute the transformation matrix; the new position of e; is the first column 
of A, and the new position of ê is the second column of A. Therefore, by looking at 
the figure, we can deduce that 

1 —1 
A= T 


6At least, A is close to that. The square corner could actually be at the point (1.01, .99). 
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To find where the point (2, 3) is sent, simply multiply 


1 —1]f2] [-1 
1 2 3; | 8 I- 
There is another way of doing this which isn’t as computational — it doesn’t involve 
computing the transformation matrix. Consider the following equalities: 


l= [e]+ G] 


This last equality states something that is somewhat obvious: to arrive at the vector 
2 — oe yeahs roe mir, once 2 
| 3 | , one needs to go 2 units in the e; direction and 3 units in the e; direction. To find 


where the point (2, 3) is transformed, one needs to go 2 units in the new e; direction 
and 3 units in the new & direction. This is demonstrated in Figure 5.6. 


y 
new location of 4 
the point (2, 3) 


\ 
3 x “new” ez 
\ 
% 
% 
w: 
\ 
\ 


77 
“2x “new” êi 
7 

+ 

X 


Y 


Figure 5.6: Finding the new location of the point (2,3). 


We are coming to grips with how matrix transformations work. We asked two ba- 
sic questions: “How do we find the matrix for a given transformation?” and “How 
do we understand the transformation without the matrix?”, and we’ve answered each 
accompanied by one example. Let’s do another example that demonstrates both tech- 
niques at once. 


Example 111 First, find the matrix A that transforms the Cartesian plane by stretch- 
ing it vertically by a factor of 1.5, then stretches it horizontally by a factor of 0.5, then 
rotates it clockwise about the origin 90°. Secondly, using the new locations of ej and 
e3, find the transformed location of the point (—1, 2). 
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SOLUTION To find A, first consider the new location of e1. Stretching the plane 
vertically does not affect ei; stretching the plane horizontally by a factor of 0.5 changes 


eito ue | , and then rotating it 90° about the origin moves it to | . This is the 


(0) 
—1/2 
first column of A. 

Now consider the new location of e3. Stretching the plane vertically changes it to 
(0) 3/2 
| 3/2 | ; stretching horizontally does not affect it, and rotating 90° moves it to | 7 | ‘ 
This is then the second column of A. This gives 


Where does the point (—1,2) get sent to? The corresponding vector ra is 


found by going —1 units in the e; direction and 2 units in the e} direction. Therefore, 
the transformation will send the vector to —1 units in the new e direction and 2 units 
in the new 6; direction. This is sketched in Figure 5.7, along with the transformed unit 
square. We can also check this multiplicatively: 


[ae al [an] 


Figure 5.8 shows the effects of the transformation on another shape. 


Figure 5.7: Understanding the transformation in Example 111. 


Y 


Figure 5.8: Transforming the Cartesian plane in Example 111 215 
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Right now we are focusing on transforming the Cartesian plane — we are making 2D 
transformations. Knowing how to do this provides a foundation for transforming 3D 
space,’ which, among other things, is very important when producing 3D computer 
graphics. Basic shapes can be drawn and then rotated, stretched, and/or moved to 
other regions of space. This also allows for things like “moving the camera view.” 


What kinds of transformations are possible? We have already seen some of the 
things that are possible: rotations, stretches, and flips. We have also mentioned some 
things that are not possible. For instance, we stated that straight lines always get trans- 
formed to straight lines. Therefore, we cannot transform the unit square into a circle 
using a matrix. 


Let’s look at some common transformations of the Cartesian plane and the matri- 
ces that perform these operations. In the following figures, a transformation matrix 
will be given alongside a picture of the transformed unit square. (The original unit 
square is drawn lightly as well to serve as a reference.) 


2D Matrix Transformations 


y 
Horizontal stretch by a 
factor of k. (k, 1) 
k 0 
O 1 
x 
y 
Vertical stretch by a (1, k) 
factor of k. 
1 0 
O k 
< t > X 


7 Actually, it provides a foundation for doing it in 4D, 5D, . . ., 17D, etc. Those are just harder to visualize. 
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Horizontal shear by a T 
factor of k. 


[o 3] Z 


Vertical shear by a factor 


of k. 
1 0 
k 1 


Horizontal reflection 
across the y axis. 


-1 0 ~ =x 
[o il 


Vertical reflection across 
the x axis. 


1 0 X 
0 -1 
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y 
A 
Diagonal reflection 
across the line y = x. 
0 1 A 
1 0 
Y 
y 


Rotation around the Hu 
origin by an angle of 8. 


cos@ —sin@ 
sinf  cosé 


Projection onto the x y 
axis. + 
(Note how the square is 
“squashed” down onto t.- 

the x-axis.) x 


fo ol | 


Projection onto the y y 

axis. 

(Note how the square is 

“squashed” over onto 

the y-axis.) < > xX 


fo 1] 


Now that we have seen a healthy list of transformations that we can perform on 
the Cartesian plane, let’s practice a few more times creating the matrix that gives the 
desired transformation. In the following example, we develop our understanding one 
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more critical step. 


Example 112 Find the matrix A that transforms the Cartesian plane by perform- 
ing the following operations in order: 


1. Vertical shear by a factor of 3. Horizontal stretch by a fac- 


0.5 tor of 2 
2. Counterclockwise rotation 4. Diagonal reflection across 
about the origin by an angle the line y = x 
of 0 = 30° 
SOLUTION Wow! We already know how to do this — sort of. We know we 


can find the columns of A by tracing where ei and & end up, but this also seems 
difficult. There is so much that is going on. Fortunately, we can accomplish what we 
need without much difficulty by being systematic. 

First, let’s perform the vertical shear. The matrix that performs this is 


1 0 
fp= bee aE 


After that, we want to rotate everything clockwise by 30°. To do this, we use 


cos 30° —sin30° /3/2 —1/2 
A=] .. an iE : 
sin30° cos 30 1/2 3/2 


In order to do both of these operations, in order, we multiply A2A1.8 
To perform the final two operations, we note that 


2 0 01 
m= la | and a= | 


perform the horizontal stretch and diagonal reflection, respectively. Thus to perform 
all of the operations “at once,” we need to multiply by 


A = AgA3A2Ai 
[e ollo a o ellos 1 
: iver ki 


2v3 — 1)/2 -1 
_ [0.933 0.866 
1.232 —1 


8The reader might ask, “Is it important to do multiply these in that order? Could we have multiplied 
A1A2 instead?” Our answer starts with “Is matrix multiplication commutative?” The answer to our question 
is “No,” so the answers to the reader’s questions are “Yes” and “No,” respectively. 
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Let’s consider this closely. Suppose | want to know where a vector x ends up. We 
claim we can find the answer by multiplying Ax. Why does this work? Consider: 


AX = AgA3A2A1X 


= A4A3A2(A1X) (performs the vertical shear) 

= AyA3(A2x,) (performs the rotation) 

= Aq(A3X2) (performs the horizontal stretch) 
= Axx; (performs the diagonal reflection) 
=X; (the result of transforming x) 


Most readers are not able to visualize exactly what the given list of operations does 
to the Cartesian plane. In Figure 5.9 we sketch the transformed unit square; in Figure 
5.10 we sketch a shape and its transformation. 


Figure 5.9: The transformed unit square in Example 112. 


7 


Figure 5.10: A transformed shape in Example 112. 


Once we know what matrices perform the basic transformations,” performing com- 
plex transformations on the Cartesian plane really isn’t that . . . complex. It boils down 


220 Ior know where to find them 


to multiplying by a series of matrices. 


5.1 Transformations of the Cartesian Plane 


We’ve shown many examples of transformations that we can do, and we’ve men- 
tioned just a few that we can’t — for instance, we can’t turn a square into a circle. Why 
not? Why is it that straight lines get sent to straight lines? We spent a lot of time within 
this text looking at invertible matrices; what connections, if any,!? are there between 
invertible matrices and their transformations on the Cartesian plane? 

All these questions require us to think like mathematicians — we are being asked to 
study the properties of an object we just learned about and their connections to things 
we’ve already learned. We’ll do all this (and more!) in the following section. 


Exercises 5.1 


In Exercises 1 — 4, a sketch of transformed unit In Exercises 5 — 10, a list of transformations is 
square is given. Find the matrix A that per- given. Find the matrix A that performs those 


forms this transformation. 


transformations, in order, on the Cartesian 


y plane. 
4 5. (a) vertical shear by a factor of 2 
(b) horizontal shear by a factor of 2 
6. (a) horizontal shear by a factor of 2 
(b) vertical shear by a factor of 2 
7. (a) horizontal stretch by a factor of 3 
(b) reflection across the line y = x 
1. 8. (a) counterclockwise rotation by an 
angle of 45° 
(b) vertical stretch by a factor of 1/2 
y 
4 9. (a) clockwise rotation by an angle of 
90° 
(b) horizontal reflection across the y 
2. axis 
1 a (c) vertical shear by a factor of 1 
Y 
10. (a) vertical reflection across the x 
A axis 
(b) horizontal reflection across the y 
axis 
3. 1 (c) diagonal reflection across the line 
x y=x 
1 1 
In Exercises 11 — 14, two sets of transforma- 
y tions are given. Sketch the transformed unit 
4. 1 4 square under each set of transformations. 
A Ae x Are the transformations the same? Explain 
1 why/why not. 


10By now, the reader should expect connections to exist. 
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11. (a) a horizontal reflection across the 13. (a) a horizontal stretch by a factor of 
y axis, followed by a vertical re- 1/2 followed by a vertical stretch 
flection across the x axis, com- by a factor of 3, compared to 
pared to (b) the same operations but in oppo- 

(b) a counterclockise rotation of site order 
180° 14. (a) a reflection across the line y = 


x followed by a reflection across 


12. (a) a horizontal stretch by a factor of the x axis, compared to 


2 followed by a reflection across 


the line y = x, compared to (b) a reflection across the the y axis, 
, followed by a reflection across 
(b) a vertical stretch by a factor of 2 the line y = x. 


5.2 Properties of Linear Transformations 


AS YOU READ 


1. T/F: Translating the Cartesian plane 2 units up is a linear transformation. 


2. T/F: If Tis a linear transformation, then T(0) = 0, 


In the previous section we discussed standard transformations of the Cartesian 
plane — rotations, reflections, etc. As a motivational example for this section’s study, 
let’s consider another transformation — let’s find the matrix that moves the unit square 
one unit to the right (see Figure 5.11). This is called a translation. 


y y 
A. x on x 
Figure 5.11: Translating the unit square one unit to the right. 
Our work from the previous section allows us to find the matrix quickly. By looking 


3 na a 2 ai 1 
at the picture, it is easy to see that ei is moved to 0 and e is moved to il 


Therefore, the transformation matrix should be 


However, look at Figure 5.12 where the unit square is drawn after being trans- 
formed by A. Itis clear that we did not get the desired result; the unit square was not 
translated, but rather stretched/sheared in some way. 


5.2 Properties of Linear Transformations 


Figure 5.12: Actual transformation of the unit square by matrix A. 


What did we do wrong? We will answer this question, but first we need to develop 
a few thoughts and vocabulary terms. 

We’ve been using the term “transformation” to describe how we’ve changed vec- 
tors. In fact, “transformation” is synonymous to “function.” We are used to functions 
like f(x) = x’, where the input is a number and the output is another number. In the 
previous section, we learned about transformations (functions) where the input was 
a vector and the output was another vector. If A is a “transformation matrix,” then we 
could create a function of the form T(x) = Ax. That is, a vector x is the input, and the 
output is X multiplied by A.1" 

When we defined f(x) = x? above, we let the reader assume that the input was 
indeed a number. If we wanted to be complete, we should have stated 


f:R>R where f(x) =x’. 


The first part of that line told us that the input was a real number (that was the first 
R) and the output was also a real number (the second R). 

To define a transformation where a 2D vector is transformed into another 2D vector 
via multiplication by a 2 x 2 matrix A, we should write 


T:R? >R? where T(x) = Ax. 
Here, the first R? means that we are using 2D vectors as our input, and the second R? 


means that a 2D vector is the output. 
Consider a quick example: 


2 
xy 
T:R? > R? where AN 2x, 


Notice that this takes 2D vectors as input and returns 3D vectors as output. For in- 


stance, 
9 
eRe 
—6 


We now define a special type of transformation (function). 


11We used T instead of f to define this function to help differentiate it from “regular” functions. “Nor- 
mally” functions are defined using lower case letters when the input is a number; when the input is a vector, 
we use upper case letters. 
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Definition 33 


Linear Transformation 


A transformation T : R” — R” is a linear transformation if 
it satisfies the following two properties: 


T(X + Y) = T(X) + T(y) for all vectors X and y, and 


T(kX) = kT(X) for all vectors x and all scalars k. 


If Tis a linear transformation, it is often said that “Tis linear.” 


Let’s learn about this definition through some examples. 


Example 113 


Determine whether or not the transformation T : R? > R? isa 


linear transformation, where 


SOLUTION 


We'll arbitrarily pick two vectors X and y: 


ee] = s-i] 


Let’s check to see if T is linear by using the definition. 


1. Is T(x + y) = T(x) + T(y)? First, compute X 


aye) | i- pi 


Now compute 7(x), T(y), and T(x + y): 
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Do w=) weai) 


1 
5 


9 1] , [16 
6|+]2/4]|8 
—6 5 12 


Therefore, Tis not a linear transformation. 


5.2 Properties of Linear Transformations 


So we have an example of something that doesn’t work. Let’s try an example where 
things do work.’ 


Example 114 Determine whether or not the transformation T : R? — R? is a 
linear transformation, where T(x) = AX and 
1 2 
A= E A | 
SOLUTION Let’s start by again considering arbitrary x and y. Let’s choose the 


same x and y from Example 113. 


[2] = =E] 


If the lineararity properties hold for these vectors, then maybe it is actually linear (and 
we'll do more work). 


1. Is T(X + y) = T(X) + T(y)? Recall: 


>, > 4 
X+y= He 


Now compute T(X), T(y), and T(x) + T(y): 


II 
m~a 
m | 

m.e 
l 

II 
= 
N e 
w e 
| A |! 

II 
m 
N e 
A O 
——} 


Is T(X + Y) = T(X) + TV)? 


So far, so good: T(x + y) is equal to T(x) + T(y). 


12Recall a principle of logic: to show that something doesn’t work, we just need to show one case where 
it fails, which we did in Example 113. To show that something a/ways works, we need to show it works 
for all cases — simply showing it works for a few cases isn’t enough. However, doing so can be helpful in 
understanding the situation better. 
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2. Is T(kx) = kT(X)? Let’s arbitrarily pick k = 7, and use x as before. 


ron=r(| 1) 
[E] 


=7: T8) | 


So far it seems that T is indeed linear, for it worked in one example with arbitrarily 
chosen vectors and scalar. Now we need to try to show it is always true. 
Consider T(x + y). By the definition of T, we have 


T(X+Y) = A(X +y). 
By Theorem 3, part 2 (on page 66) we state that the Distributive Property holds for 


matrix multiplication.’ So A(x + y) = AX + Ay. Recognize now that this last part is 
just T(x) + T(y)! We repeat the above steps, all together: 


T(x +Y) = A(X +y) (by the definition of T in this example) 
= AX + Ay (by the Distributive Property) 
= T(x) + T(y) (again, by the definition of 7) 


Therefore, no matter what X and y are chosen, T(x + y) = T(x) + T(y). Thus the first 
part of the lineararity definition is satisfied. 
The second part is satisfied in a similar fashion. Let k be a scalar, and consider: 


T(kx) = A(kx) (by the definition of T is this example) 
= kAX (by Theorem 3 part 3) 
= kT(X) (again, by the definition of T) 


Since T satisfies both parts of the definition, we conclude that T is a linear trans- 
formation. 


We have seen two examples of transformations so far, one which was not linear and 
one that was. One might wonder “Why is linearity important?”, which we’ll address 
shortly. 

First, consider how we proved the transformation in Example 114 was linear. We 
defined T by matrix multiplication, that is, T(x) = Ax. We proved T was linear using 
properties of matrix multiplication — we never considered the specific values of A! That 
is, we didn’t just choose a good matrix for T; any matrix A would have worked. This 


13 Recall that a vector is just a special type of matrix, so this theorem applies to matrix-vector multiplica- 
tion as well. 
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leads us to an important theorem. The first part we have essentially just proved; the 
second part we won’t prove, although its truth is very powerful. 


Theorem 23 Matrices and Linear Transformations 


1. Define T : R” > R” by T(x) = AX, where A is an 
m x n matrix. Then Tis a linear transformation. 


2. Let T : R” — R” be any linear transformation. Then 


there exists an unique m xn matrix A such that T(x) 
Ax. 


The second part of the theorem says that all linear transformations can be de- 
scribed using matrix multiplication. Given any linear transformation, there is a matrix 
that completely defines that transformation. This important matrix gets its own name. 


Definition 34 Standard Matrix of a Linear Transformation 


Let T : R” — R” bea linear transformation. By Theorem 
23, there is a matrix A such that T(x) = AX. This matrix A 


is called the standard matrix of the linear transformation T, 
and is denoted [T]. 


"The matrix—like brackets around T suggest that the standard matrix A 
is a matrix “with T inside.” 


While exploring all of the ramifications of Theorem 23 is outside the scope of this 
text, let it suffice to say that since 1) linear transformations are very, very important 
in economics, science, engineering and mathematics, and 2) the theory of matrices is 
well developed and easy to implement by hand and on computers, then 3) it is great 
news that these two concepts go hand in hand. 


We have already used the second part of this theorem in a small way. In the pre- 
vious section we looked at transformations graphically and found the matrices that 
produced them. At the time, we didn’t realize that these transformations were linear, 
but indeed they were. 


This brings us back to the motivating example with which we started this section. 
We tried to find the matrix that translated the unit square one unit to the right. Our 
attempt failed, and we have yet to determine why. Given our link between matrices 
and linear transformations, the answer is likely “the translation transformation is not 
a linear transformation.” While that is a true statement, it doesn’t really explain things 
all that well. Is there some way we could have recognized that this transformation 
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wasn’t linear?!’ 
Yes, there is. Consider the second part of the linear transformation definition. It 


states that T(kX) = kT(x) for all scalars k. If we let k = 0, we have T(Ox) = 0 - T(x), or 
more simply, T(0) = 0. That is, if Tis to be a linear transformation, it must send the 
zero vector to the zero vector. 

This is a quick way to see that the translation transformation fails to be linear. By 
shifting the unit square to the right one unit, the corner at the point (0,0) was sent to 
the point (1, 0), i.e., 


9 was sent to the vector 1 
(0) (00 


the vector | 


This property relating to Õis important, so we highlight it here. 


Key Idea 15 Linear Transformations and 0 


Let T: R” — R” bea linear transformation. Then: 


= 


T(n) = Om. 


That is, the zero vector in IR" gets sent to the zero vector in 
IR, 


The interested reader may wish to read the footnote below.?° 


The Standard Matrix of a Linear Transformation 


It is often the case that while one can describe a linear transformation, one doesn’t 
know what matrix performs that transformation (i.e., one doesn’t know the standard 
matrix of that linear transformation). How do we systematically find it? We’ll need a 


new definition. 


Definition 35 Standard Unit Vectors 


In IR", the standard unit vectors é; are the vectors with a 1 
in the i” entry and Os everywhere else. 


M4 That is, apart from applying the definition directly? 

15 The idea that linear transformations “send zero to zero” has an interesting relation to terminology. The 
reader is likely familiar with functions like f(x) = 2x + 3 and would likely refer to this as a “linear function.” 
However, f(0) 4 0, so f is not “linear” by our new definition of linear. We erroneously call f “linear” since 
its graph produces a line, though we should be careful to instead state that “the graph of fis a line.” 
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We’ve already seen these vectors in the previous section. In R?, we identified 


= 1 $ (0) 
a= fa] and a= [2]: 


In R4, there are 4 standard unit vectors: 


Di 
ll 
ALT 
ll 
Oy 
| 


, and e= 


OoOOORFR 
OoOOrRO 
OorROO 
rPOoOO°O 


How do these vectors help us find the standard matrix of a linear transformation? 
Recall again our work in the previous section. There, we practiced looking at the trans- 
formed unit square and deducing the standard transformation matrix A. We did this 
by making the first column of A the vector where e; ended up and making the second 
column of A the vector where e3 ended up. One could represent this with: 


That is, T(€;) is the vector where €; ends up, and T(e3) is the vector where ê, ends up. 

The same holds true in general. Given a linear transformation T : R” — R”, the 
standard matrix of T is the matrix whose it column is the vector where é; ends up. 
While we won’t prove this is true, it is, and it is very useful. Therefore we’ll state it 
again as a theorem. 


Theorem 24 The Standard Matrix of a Linear Transformation 


Let T : R” — R” bea linear transformation. Then |T] is the 


m x n matrix: 


Let’s practice this theorem in an example. 


Example 115 Define T : R? —> R* to be the linear transformation where 
X +X 
xı 1 2 
3x1 — X3 
T X2 = 
2x2 + 5X3 
X3 


4xı + 3x2 + 2x3 
Find [T]. 
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SOLUTION T takes vectors from R? into R’, so [ T] is going to be a 4 x 3 matrix. 
Note that 
1 0 0 
e&=]0], ex =] 1 and e3=1]|0 
0 0 1 
We find the columns of |T] by finding where ei, êb and e3 are sent, that is, we find 


T(€1), T(e3) and T(é3). 


1 0 0 
T(é1) =T 0 T(&) =T 1 T(é3) =T 0 
0 0 1 
H H bel 
13 _ 10 = 4-1 
~ 10 ~ 12 a) 
4 3 2 
Thus 
1 1 0 
3 0 —1 
Tas 02 5 
43 2 
Let’s check this. Consider the vector 
1 
X= 12 
3 


Strictly from the original definition, we can compute that 


r 1+2 3 
7 3—3 0 
T(x) =T k 4+15 | |19 
4+6+6 16 


Now compute T(x) by computing [ T]x= AX. 


TEI 3 
, 13 0 -1 0 
PEN ig 5 et f 19 
43 2 16 


They match!*¢ 


Let’s do another example, one that is more application oriented. 


16 Of course they do. That was the whole point. 
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Example 116 A baseball team manager has collected basic data concerning his 
hitters. He has the number of singles, doubles, triples, and home runs they have hit 
over the past year. For each player, he wants two more pieces of information: the total 
number of hits and the total number of bases. 


Using the techniques developed in this section, devise a method for the manager 
to accomplish his goal. 


SOLUTION If the manager only wants to compute this for a few players, then 
he could do it by hand fairly easily. After all: 


total # hits = # of singles + # of doubles + # of triples + # of home runs, 


and 


total # bases = # of singles + 2x # of doubles + 3 x# of triples + 4x# of home runs. 


However, if he has a lot of players to do this for, he would likely want a way to 
automate the work. One way of approaching the problem starts with recognizing that 
he wants to input four numbers into a function (i.e., the number of singles, doubles, 
etc.) and he wants two numbers as output (i.e., number of hits and bases). Thus he 
wants a transformation T : R — R? where each vector in R* can be interpreted as 


# of singles 
# of doubles 
# of triples 
# of home runs 


and each vector in R? can be interpreted as 
# of hits 
# of bases | ` 


To find | T], he computes T(e;), T(e3), T(e3) and T(eq). 
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1 
na\=7 i T(&) =T e 
0 
1 
1 p= 
a B 
0 
0 m6) =T]} | 5 
T(é3) =T 0 
a 1 


1 
| means that 


(What do these calculations mean? For example, finding T(e3) = | 3 


one triple counts as 1 hit and 3 bases.) 


Thus our transformation matrix [T Jis 


As an example, consider a player who had 102 singles, 30 doubles, 8 triples and 14 
home runs. By using A, we find that 


meaning the player had 154 hits and 242 total bases. 


A question that we should ask concerning the previous example is “How do we 
know that the function the manager used was actually a linear transformation? After 
all, we were wrong before — the translation example at the beginning of this section 
had us fooled at first.” 

This is a good point; the answer is fairly easy. Recall from Example 113 the trans- 
formation 


5.2 Properties of Linear Transformations 


and from Example 115 


X1 T X2 
T. 1 _ 3x1 — X3 
115 X2 = 2x2 + 5x3 , 
X3 


4xı + 3x2 + 2x3 


where we use the subscripts for T to remind us which example they came from. 

We found that T113 was not a linear transformation, but stated that T115 was (al- 
though we didn’t prove this). What made the difference? 

Look at the entries of T113(X) and Tias (X). T113 contains entries where a variable is 
squared and where 2 variables are multiplied together — these prevent T113 from being 
linear. On the other hand, the entries of T115 are all of the form a4x; +--+ -+ anXn; that 
is, they are just sums of the variables multiplied by coefficients. Tis linear if and only if 
the entries of T(x) are of this form. (Hence linear transformations are related to linear 
equations, as defined in Section 1.1.) This idea is important. 


Key Idea 16 Conditions on Linear Transformations 


Let T : R” — R” be a transformation and consider the 


entries of 
X1 


X2 
NEO) =T 


Xn 


Tis linear if and only if each entry of T(x) is of the form a,x, + 
A2X2 + ` -` AnXn. 


Going back to our baseball example, the manager could have defined his transfor- 
mation as 


Xı 

T X2 z Xı + X2 + X3 + X4 
X3 Xı + 2X2 + 3X3 + 4X4 
X4 


Since that fits the model shown in Key Idea 16, the transformation T is indeed linear 
and hence we can find a matrix | T] that represents it. 
Let’s practice this concept further in an example. 


Example 117 Using Key Idea 16, determine whether or not each of the following 
transformations is linear. 


aD- ee ED el 
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nae E] 
X2 TX2 

SOLUTION Tı is not linear! This may come as a surprise, but we are not al- 
lowed to add constants to the variables. By thinking about this, we can see that this 
transformation is trying to accomplish the translation that got us started in this section 
— it adds 1 to all the x values and leaves the y values alone, shifting everything to the 
right one unit. However, this is not linear; again, notice how Ö does not get mapped 
to 0. 

Tə is also not linear. We cannot divide variables, nor can we put variabless in- 
side the square root function (among other other things; again, see Section 1.1). This 
means that the baseball manager would not be able to use matrices to compute a 
batting average, which is (number of hits)/(number of at bats). 

T is linear. Recall that \/7 and 7 are just numbers, just coefficients. 


We’ve mentioned before that we can draw vectors other than 2D vectors, although 
the more dimensions one adds, the harder it gets to understand. In the next section 
we'll learn about graphing vectors in 3D — that is, how to draw on paper or a computer 
screen a 3D vector. 


Exercises 5.2 


In Exercises 1 — 5, a transformation T is given. xı Xı + 2x2 
Determine whether or not T is linear; if not, 7. T | D = 


state why not. ( 
1. r([*]) = [ x1 +x2 | 
| X2 | 3x1 — X2 
iy PERY. 8. T 
aR 
| X2 | X1 — X2 
[x = [x +1 
X = |yo41 x1 X1 + 3x3 
~ ` 9. T X2 = X1 — X3 


Xi + 2X2 = 3x3 
0 

Xi + 4x3 
5x2 + X3 


w 
ans | 


4. r([*]) = [3] X3 X1 + X3 
| X2 L 1 
r f 0 
=) -f sak 
er al) =le 
In Exercises 6 — 11, a linear transformation T 
is given. Find | 7]. 


= [xı + 2x2 + 3x3 + 4xa | 


i oe 11. T 
endale] 
X2 | X1 — X2 
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5.3 Visualizing Vectors: Vectors in Three Dimensions 


AS YOU READ 


1. T/F: The viewpoint of the reader makes a difference in how vectors in 3D look. 


2. T/F: If two vectors are not near each other, then they will not appear to be near 
each other when graphed. 


3. T/F: The parallelogram law only applies to adding vectors in 2D. 


We ended the last section by stating we could extend the ideas of drawing 2D vec- 
tors to drawing 3D vectors. Once we understand how to properly draw these vectors, 
addition and subtraction is relatively easy. We’ll also discuss how to find the length of 
a vector in 3D. 

We start with the basics of drawing a vector in 3D. Instead of having just the tradi- 
tional x and y axes, we now add a third axis, the z axis. Without any additional vectors, 
a generic 3D coordinate system can be seen in Figure 5.13. 


X 


Figure 5.13: The 3D coordinate system 


In 2D, the point (2, 1) refers to going 2 units in the x direction followed by 1 unit in 
the y direction. In 3D, each point is referenced by 3 coordinates. The point (4, 2, 3) is 
found by going 4 units in the x direction, 2 units in the y direction, and 3 units in the z 
direction. 

How does one sketch a vector on this coordinate system? As one might expect, we 

1 
can sketch the vector V = | 2 | by drawing an arrow from the origin (the point (0,0,0)) 
3 
to the point (1, 2, 3)? The only “tricky” part comes from the fact that we are trying 
to represent three dimensional space on a two dimensional sheet of paper. However, 


170f course, we don’t have to start at the origin; all that really matters is that the tip of the arrow is 1 unit 
in the x direction, 2 units in the y direction, and 3 units in the z direction from the origin of the arrow. 
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it isn’t really hard. We’ll discover a good way of approaching this in the context of an 
example. 


Example 118 Sketch the following vectors with their origin at the origin. 
2 1 
v= ]1 and ü= | 3 
3 —1 
SOLUTION We'll start with V first. Starting at the origin, move 2 units in the x 


direction. This puts us at the point (2,0,0) on the x axis. Then, move 1 unit in the y 
direction. (In our method of drawing, this means moving 1 unit directly to the right. 
Of course, we don’t have a grid to follow, so we have to make a good approximation of 
this distance.) Finally, we move 3 units in the z direction. (Again, in our drawing, this 
means going straight “up” 3 units, and we must use our best judgment in a sketch to 
measure this.) 

This allows us to locate the point (2, 1,3); now we draw an arrow from the origin 
to this point. In Figure 5.14 we have all 4 stages of this sketch. The dashed lines show 
us moving down the x axis in (a); in (b) we move over in the y direction; in (c) we move 
up in the z direction, and finally in (d) the arrow is drawn. 


Z Z 


(d) 


Figure 5.14: Stages of sketching the vector V for Example 118. 


5.3 Visualizing Vectors: Vectors in Three Dimensions 


Drawing the dashed lines help us find our way in our representation of three di- 
mensional space. Without them, it is hard to see how far in each direction the vector 
is supposed to have gone. 

To draw U, we follow the same procedure we used to draw V. We first locate the 
point (1,3, —1), then draw the appropriate arrow. In Figure 5.15 we have @ drawn 
along with v. We have used different dashed and dotted lines for each vector to help 
distinguish them. 

Notice that this time we had to go in the negative z direction; this just means we 
moved down one unit instead of up a unit. 


Figure 5.15: Vectors V and u in Example 118. 


As in 2D, we don’t usually draw the zero vector, 


It doesn’t point anywhere. It is a conceptually important vector that does not have a 
terribly interesting visualization. 

Our method of drawing 3D objects on a flat surface — a 2D surface —is pretty clever. 
It isn’t perfect, though; visually, drawing vectors with negative components (especially 
negative x coordinates) can look a bit odd. Also, two very different vectors can point 
to the same place. We’ll highlight this with our next two examples. 


—3 
Example 119 Sketch the vector V = | —1 
2 
SOLUTION We use the same procedure we used in Example 118. Starting at 


the origin, we move in the negative x direction 3 units, then 1 unit in the negative y 
direction, and then finally up 2 units in the z direction to find the point (—3, —1, 2). 
We follow by drawing an arrow. Our sketch is found in Figure 5.16; V is drawn in two 
coordinate systems, once with the helpful dashed lines, and once without. The second 
drawing makes it pretty clear that the dashed lines truly are helpful. 
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>» y 
Figure 5.16: Vector V in Example 119. 
2 —2 
Example 120 Draw the vectors V = | 4 | and = | 1 | on the same coordi- 
2 —1 
nate system. 
SOLUTION We follow the steps we’ve taken before to sketch these vectors, 


shown in Figure 5.17. The dashed lines are aides for V and the dotted lines are aids 
for U. We again include the vectors without the dashed and dotted lines; but without 
these, it is very difficult to tell which vector is which! 


Figure 5.17: Vectors V and u in Example 120. 


Our three examples have demonstrated that we have a pretty clever, albeit imper- 
fect, method for drawing 3D vectors. The vectors in Example 120 look similar because 
of our viewpoint. In Figure 5.18 (a), we have rotated the coordinate axes, giving the 
vectors a different appearance. (Vector V now looks like it lies on the y axis.) 

Another important factor in how things look is the scale we use for the x, y, and 
z axes. In 2D, it is easy to make the scale uniform for both axes; in 3D, it can be a bit 
tricky to make the scale the same on the axes that are “slanted.” Figure 5.18 (b) again 
shows the same 2 vectors found in Example 120, but this time the scale of the x axis 


5.3 Visualizing Vectors: Vectors in Three Dimensions 


is a bit different. The end result is that again the vectors appear a bit different than 
they did before. These facts do not necessarily pose a big problem; we must merely 
be aware of these facts and not make judgments about 3D objects based on one 2D 
: 18 

image. 


(a) (b) 


Figure 5.18: Vectors V and d in Example 120 with 
a different viewpoint (a) and x axis scale (b). 


We now investigate properties of vector arithmetic: what happens (i.e., how do 
we draw) when we add 3D vectors and multiply by a scalar? How do we compute the 
length of a 3D vector? 


Vector Addition and Subtraction 


In 2D, we saw that we could add vectors together graphically using the Parallelo- 
gram Law. Does the same apply for adding vectors in 3D? We investigate in an example. 


2 1 
Example 121 Let V= | 1| andu= | 3 |. Sketchv+u. 
3 —1 
SOLUTION We sketched each of these vectors previously in Example 118. We 
3 
sketch them, along with V+ü = | 4 |, in Figure 5.19 (a). (We use loosely dashed lines 
2 


for V+ ü.) 


18The human brain uses both eyes to convey 3D, or depth, information. With one eye closed (or missing), 
we can have a very hard time with “depth perception.” Two objects that are far apart can seem very close 
together. A simple example of this problem is this: close one eye, and place your index finger about a foot 
above this text, directly above this WORD. See if you were correct by dropping your finger straight down. 
Did you actually hit the proper spot? Try it again with both eyes, and you should see a noticable difference 
in your accuracy. 

Looking at 3D objects on paper is a bit like viewing the world with one eye closed. 
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Does the Parallelogram Law still hold? In Figure 5.19 (b), we draw additional rep- 
resentations of V and d to form a parallelogram (without all the dotted lines), which 
seems to affirm the fact that the Parallelogram Law does indeed hold. 


(b) 


Figure 5.19: Vectors V, U, and V + U Example 121. 


We also learned that in 2D, we could subtract vectors by drawing a vector from the 
tip of one vector to the other.’? Does this also work in 3D? We’ll investigate again with 
an example, using the familiar vectors V and u from before. 


2 1 
Example 122 Let V= |1| andū= | 3 |. Sketchv¥—Jd. 
3 —1 
1 
SOLUTION It is simple to compute that ¥ — G@ = | —2 |. All three of these 
4 


vectors are sketched in Figure 5.20 (a), where again Vis guided by the dashed, U by the 
dotted, and V — du by the loosely dashed lines. 


Does the 2D subtraction rule still hold? That is, can we draw V — ü by drawing an 
arrow from the tip of Ū to the tip of V? In Figure 5.20 (b), we translate the drawing of 
V— ü to the tip of U, and sure enough, it looks like it works. (And in fact, it really does.) 


19Recall that it is important which vector we used for the origin and which was used for the tip. 
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(b) 


Figure 5.20: Vectors V, U, and V — u from Example 122. 


The previous two examples highlight the fact that even in 3D, we can sketch vec- 
tors without explicitly knowing what they are. We practice this one more time in the 
following example. 


Example 123 Vectors Vand Uare drawn in Figure 5.21. Using this drawing, sketch 
the vectors V+ ü and V — U. 


Sl 


Figure 5.21: Vectors V and u for Example 123. 


SOLUTION Using the Parallelogram Law, we draw v + ü by first drawing a gray 
version of U coming from the tip of V; V + dis drawn dashed in Figure 5.22. 


To draw V — ü, we draw a dotted arrow from the tip of ito the tip of V. 
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Figure 5.22: Vectors V, U, V + Gand Vv — ü for Example 123. 


Scalar Multiplication 


Given a vector V in 3D, what does the vector 27V look like? How about —V? After 
learning about vector addition and subtraction in 3D, we are probably gaining confi- 
dence in working in 3D and are tempted to say that 2V is a vector twice as long as V, 
pointing in the same direction, and —V is a vector of the same length as V, pointing 
in the opposite direction. We would be right. We demonstrate this in the following 


example. 
Example 124 Sketch V, 2V, and —v, where 
1 
V=|2 
3 
SOLUTION 


X 


Figure 5.23: Sketching scalar multiples of V in Example 124. 
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It is easy to compute 
2 

2v=|4 and —vV=]-2 
6 


These are drawn in Figure 5.23. This figure is, in many ways, a mess, with all the 
dashed and dotted lines. They are useful though. Use them to see how each vec- 
tor was formed, and note that 2V at least looks twice as long as V, and it looks like —V 
points in the opposite direction.”° 


Vector Length 


How do we measure the length of a vector in 3D? In 2D, we were able to answer this 
question by using the Pythagorean Theorem. Does the Pythagorean Theorem apply in 
3D? Ina sense, it does. 

1 
Consider the vector V = | 2 |, as drawn in Figure 5.24 (a), with guiding dashed 
3 
lines. Now look at part (b) of the same figure. Note how two lengths of the dashed 
lines have now been drawn gray, and another dotted line has been added. 


(b) 


Figure 5.24: Computing the length of v 


These gray dashed and dotted lines form a right triangle with the dotted line form- 
ing the hypotenuse. We can find the length of the dotted line using the Pythagorean 
Theorem. 


length of the dotted line = sum of the squares of the dashed line lengths 


That is, the length of the dotted line = V12 + 22 = v5. 


20Qur previous work showed that looks can be deceiving, but it is indeed true in this case. 
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Now consider this: the vector v is the hypotenuse of another right triangle: the 
one formed by the dotted line and the vertical dashed line. Again, we employ the 
Pythagorean Theorem to find its length. 


length of V = (length of dashed gray line)? + (length of black dashed line)? 


Thus, the length of V is (recall, we denote the length of v with ||v]|): 


Iivi] = (length of gray line)? + (length of black line)? 


= v5 +3 
= y5 +3? 


Let’s stop for a moment and think: where did this 5 come from in the previous 
equation? It came from finding the length of the gray dashed line — it came from 12+22?. 
Let’s substitute that into the previous equation: 


lvi] = v5 + 3? 
= 4/12 + 22 + 32 
= y 14 


The key comes from the middle equation: ||v|| = v12 + 2? + 32. Do those num- 
bers 1, 2, and 3 look familiar? They are the component values of V! This is very similar 
to the definition of the length of a 2D vector. After formally defining this, we’ll practice 
with an example. 


Definition 36 3D Vector Length 


Let 
X41 
X2 
X3 


The length of V, denoted ||v||, is 
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Example 125 Find the lengths of vectors V and ü, where 
2 —4 
v= |-3 and u=| 7 
5 0 
SOLUTION We apply Definition 36 to each vector: 
||V|| = V2? + (—3)? + 5? 
= f/44+9425 
lal = Va ee eo 
= V16 + 49 
= v65 


Here we end our investigation into the world of graphing vectors. Extensions into 
graphing 4D vectors and beyond can be done, but they truly are confusing and not 
really done except for abstract purposes. 

There are further things to explore, though. Just as in 2D, we can transform 3D 
space by matrix multiplication. Doing this properly — rotating, stretching, shearing, 
etc. — allows one to manipulate 3D space and create incredible computer graphics. 


Exercises 5.3 


In Exercises 1 — 4, vectors X and y are given. In Exercises 5 — 8, vectors X and y are drawn. 
Sketch x, y, X+y, and ¥— y on the same Carte- Sketch 2x, —y, X + y, and X — ý on the same 


sian axes. Cartesian axes. 
1 2 
1. X= | -1 |,ýy= | 3 
2 2 
2 =| A 
2. x=]| 4 |,y= | -3 
—1 —1 A 
1 3 
3. x=]|1ļ|,y=]ļ3 5. 
2 6 y y 
0 0 
4.x=|1|,y=]|-1 
1 1 x 


245 


Chapter 5 Graphical Explorations of Vectors 


z y Z 
y 
6. 8: z 
y y 
x 
x 
a In Exercises 9 — 12, a vector X and a scalar 
a are given. Using Definition 36, compute 
the lengths of x and ax, then compare these 
lengths. 
1 
9.X=|-2|,a=2 
5 
-3 
5 10. x= | 4 |,a=-1 
3 
> 7 
y a 
7. 11. X= |2|,a=5 
y 1 
1 
12. X= | 2 |,a=3 
x A 2 
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COMPLEX NUMBERS 


A.1 Complex Numbers and Arithmetic 


AS YOU READ 


1. Are there any familiar arithmetic properties for the reals that don’t work for the 
complex numbers? 


. What is surprising about complex conjugation? 
T/F: Every real number has two “square roots.” 


T/F: Every polynomial has complex conjugate roots. 


u pe WN 


What common arithmetic operation does Euler’s Identity define? 


Motivation 


Much of the historical progression of mathematics revolves around increasingly gen- 
eral notions of what a “number” is. Whole numbers, integers, rational numbers, and 
real numbers—each extends our definition of “number” and allows us to solve new 
types of problems that were previously inaccessible. Such is the case with complex 
numbers. 

One primary motivation for using complex numbers will be to solve polynomial 
equations, many of which have no solutions if we confine our attention to the real 
numbers. For example, the equation x? — 4x + 2 = 0 has roots x = 4#¥2 = 2 + V7, 


2 
via the quadratic formula. But applying the quadratic formula to the equation 


xX —4x+5=0 


produces x = = 3 =4_ If we restrict our universe to the real numbers, this makes no 


sense, since there is no real number a such that a? = —4 (hence no meaning to “\/—4” 
as a real number). 
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Definition of the Complex Numbers 


We are thus motivated to try to make sense of expressions involving square roots of 
negative real numbers. In particular, we define a quantity “f” such that 


P= -—1 (A.1) 


and sometimes informally write “i = y —1.” Be warned, however, that the notation is 
not entirely standard. Electrical engineers traditionally use “j” for this quantity; other 
fields may use i, I, j, i or j. We will use ito extend our definition of what constitutes a 
“number.” 


Definition 37 Complex Number 


A complex number is a quantity z of the form 


z=qd + bi 


where a and b are real numbers. 


We refer to a in (A.2) as the real part of z and b as the imaginary part, and write 
Re(z) = a and Im(z) = b. (A.3) 


A complex number as in (A.2), in which the real and imaginary parts are specified 
explicitly, is said to be in rectangular form. A complex number effectively has two 
parts to it — the real part and the imaginary part. As we'll soon see, z in this form can 
be identified in a natural way with a point or vector in rectangular coordinates. 


Example 126 Find the real and imaginary parts of the complex number z = 
2+ 3i. 
SOLUTION We have Re(z) = 2 and Im(z) = 3. Note that the imaginary part 


does not include the /. 


In particular, note that, as in the above example, the imaginary part is always a real 
number. A complex number z = a + bi is said to be real if b = 0 and purely imaginary 
ifa = 0. Every real number a is thus naturally a complex number too, of the form 
a + Oj. The symbol C is used to denote the complex numbers. 


Complex Arithmetic 


As defined above, complex numbers are little more than ordered pairs of real numbers, 
e.g., 2 + 3i might just as well be written as a point “(2,3)” or a vector “(2,3)” How- 
ever, some basic operations for the real numbers—multiplication and division—can 
be extended to the complex numbers in a way that doesn’t work for general vectors. 


A.1 Complex Numbers and Arithmetic 


Complex Addition and Subtraction 


Definition 38 Complex Addition 


If z = a + bi and w = c+ di are complex numbers in rect- 


angular form, we define the sum z + w 


Z+w=(a+c)+(b+d)i. (A.4) 


That is, we add the real parts, and add the imaginary parts. In this regard adding 
z+wis like adding vectors, (a,b)+(c, d} = (a+c, b+d). It’s easy to check that addition 
of complex numbers is commutative (z + w = w + z) and associative ((u + w) +z = 
u + (w + z)), just as for real numbers. 

Subtraction is algebraically similar, just subtract real and imaginary parts. 


Definition 39 Complex Subtraction 
If z = a + bi and w = c + di are complex numbers in rect- 
angular form, we define the difference z — w 


Z—w=(a—c)+(b—d)i. (A.5) 


So far complex numbers have nothing on vectors. 
Example 127 Ifz = 2 + 3iand w = —1 + i compute z + w and z — w. 


SOLUTION We compute 
ztw=(24+-1)4+(341)i=14+4i 


and 
z— w = (2 — (—1)) + (3 — 1)i = 3 + 2i. 


Complex Multiplication 


Multiplication is where things get more interesting. If we think of z = a + bi and 
w = c + di as simply algebraic expressions, without regard to the nature of i, then 


zw = (a + bi)(c + di) = ac + adi + bci + bdi? 
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via middle school algebra. But if we take into account the fact that 2? = —1 then we 
have bdi? = —bd and so we have 


Definition 40 Complex Multiplication If z = a+biand w = c+diare com- 
plex numbers then the product zw is defined in rectangular 
form as 


zw = (ac — bd) + (ad + bc)i (A.6) 


Powers of z are defined as repeated products of z with itself, z? = (z)(z),z? = 


(z*)(z) = (z)(z)(z), and so on. 

It’s easy to check that complex multiplication has many of the same essential prop- 
erties that multiplication of real numbers possesses. Specifically, if u, w, z are complex 
numbers then 


e wz = zw (multiplication is commutative). 
e (uw)z = u(wz) (multiplication is associative). 
e u(w+z) = uw + uz (multiplication distributes over addition). 


These can all be checked by “brute force.” For example, if z = a + bi and w = c + di 
then the product zw is defined by (A.6), while computing wz merely reverses the roles 
of a andc, and band d. But this clearly doesn’t change the value of ac — bd or ad + bc, 
so (A.6) gives the same answer as for zw. Namely, 


zw = (a+bi)(c+di) = (ac—bd)+(ad+bc)i = (ca—bd)+(da+cb)i = (c+di)(a+bi) = wz. 


The other properties can be verified similarly. 


Example 128 Compute the product of z = 2 + 3i and w = —1 + 2i. 
SOLUTION The easiest way to actually multiply complex number is not to 
memorize the formula, but to simply “FOIL” the product, apply 2? = —1, and collect 


real and imaginary parts. Thus if z = 2 + 3i and w = —1 + 2i then 


zw = (2)(—1)+(2) (2/)-+(3/)(—1)+(3/) (2!) = —24+-4i-3/+6(7) = —2+i—6 = —8+i. 


Complex Division 


It stands to reason that if we can multiply, we should be able to divide, but it’s a bit 
trickier. Let z = a + bi,w = c + di; consider a,b,c,d as known or given. We want 
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to define the quotient u = z/w. This is equivalent to the statement that wu = z. If 


A.1 Complex Numbers and Arithmetic 


u = p + qi for some p and q then from (A.6) 


If this equals z then 


wu = (c + di)(p + qi) = (cp — dq) + (cq + dp)i. 


cp — dq = a and cq + dp =b. (A.7) 


If we consider p and q as two unknowns in (A.7) then we can solve to find 


O ac+bd nd T bc — ad 
ps p oe 


Since we took u = z/w = p + qi, we can define complex division. 


Definition 41 


Example 129 


SOLUTION 


Key Idea 17 


Square Roots | 


Square roots come up frequently and are easily dealt with. Let’s first look at a familiar 
setting. If a is a positive real number then the equation x? = a has exactly two solutions 
in x, one positive, one negative. We write x = ya for the positive solution. In the 
special case a = 0 the only solution to x? = O is x = 0, so vO = 0. The negative root 


Complex Division If z = a + bi and w = c + di are com- 
plex numbers then the quotient z/w is defined in rectangu- 


lar form as 
Z _ ac+bd , bc — ad 


w B+e PFR" 


(A.8) 


If z = 2 + 3i and w = —1 + 2i, compute z/w. 


Equation (A.8) gives 


2+3i 4 7, 
= i. 
—1+2i 5 5 


Complex Arithmetic The four basic arithmetic operations— 
addition, subtraction, multiplication, and division—are all 


defined for complex numbers. These operations possess the 
same algebraic properties as their real counterparts. 


to x? = awhena > Oisx = — ya. 
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Now consider the case when a is real but a < 0. In this case x? = a also has two 
solutions. We can write these as 


x = iy/ |a| and x = —i,/|al. (A.9) 


You can easily check that both satisfy x? = a when a < 0. For example, if x = /,/ || 
then (using the properties of complex multiplication) we have 


è =P(y/al)? = -Vla = -la| =a 


ifa <0. 

Unlike the situation in which a > 0O, in this case there is no generally accepted 
convention for which of ix/Ja] or —i,/|a| is the default choice for the “square root of 
a.” Thus, for example, /—9 = i,/| — 9| = 3i, or /—9 = —i,/| —9| = —3i are 
both valid. We have to specify which one we’re using, although in many cases it won’t 
matter, or we'll need both. 

Square roots (and other roots) of arbitrary complex numbers can also be defined, 
but we'll consider that a bit later in Section A.3. 


Conjugation 

Let’s begin by defining one of the more important operations we perform on complex 
numbers. 

Definition 42 Conjugation If z = a + bi then the conjugate of z, written Z, 


is defined in rectangular form as 


zZ=a—bi. (A.10) 


It’s easy to check that 


(A.11) 


and that z is real if and only if z = Z (this is exactly the same as saying that b = 0.) 


Example 130 If z = 2 + 3i compute Z, and use (A.11) to compute Re(z) and 
Im(z). 
SOLUTION We find Z = 2 — 3i. Also, 

Z 2 +3i 2-—3i 4 

ee Sc a 
2 2 2 
zZ-Z 2+ 3i) — (2 -3i 6i 

moat e C 8, 
2i 2 2i 
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Properties of Conjugation 


Conjugation is a common and important operation, with a number of surprising alge- 
braic properties. Specifically, conjugation commutes with complex addition, subtrac- 
tion, multiplication, and division. That is, 


z+}w=Zz4+W, 7—-w=2-W, w=7ZwW, z/w=Z/w. (A.12) 


As with the properties of complex arithmetic from Section 37, the properties of con- 
jugation are easy to check directly. For example, if z = a+ bi and w = c + di then 
wz = (ac — bd) + (ad + bc)i so that 


wz = (ac — bd) — (ad + bc)i. 


But since Z = a — bi and w = c — di we can also compute 


which is exactly wz from above. The other properties of conjugation can be similarly 
proved. An additional consequence of (A.12) is that for any power of a complex num- 
ber z, 


zZ = 7-2 -Z=2:2*'2=2" (A.13) 
Va M__ 
n copies n copies 


where we make use of “the conjugate of the product is the product of the conjugates” 
repeatedly. 


Example 131 Show that for any complex number z = a + bi the quantity zz is 
real. 


SOLUTION First, Z = a — bi. Then 
zz = a’ — abi + bai + b? = @° +b’, 


which is real since a and b are real. Note also that zZ is positive, unless a = b = 0, that 
is, unless z = 0. 


Division Via Conjugation 


Conjugation makes it a bit easier to perform complex division. If z = a + bi and 
w = c + di then we can compute the quotient z/w by multiplying through by 1 in the 
form W/W: 


z zw ac+bd+(bc-ad)i ac+bd bc—ad. 


= A A.14 
w ww c? + d2 c2 + d2 +d. ( ) 


where we made use of Example 131 to simplify ww = c? + a’, a real number. 
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Roots of Polynomials 


An important consequence of the properties of conjugation is this: if p(z) = ao + 
aıZ +--+ + anz” is an nth degree polynomial with real coefficients and a is a root of 
p (that is, p(a@) = 0) then @ is also a root of p. It is essential that the coefficients a, 
are real numbers. Here’s the proof—see if you can justify each step. We start with 
0 = p(a), so that conjugating both sides produces 


0 = p(a) 
since 0 is real, O = 0. Thus 
0 = p(a) 

=a) + d1Q + a2Q? ana” 

= ūpo + 01a + ara? anQ” 

= m +01 Q + 07 a? Gy a 

= Qo + d1 T + a Q? an a" 

= p(a). (A.15) 


So ais a root of p. If a is real this is a rather trivial statement, but if œ has a nonzero 
imaginary part this is an important (and potentially time-saving) insight. 

To illustrate, the cubic polynomial z? + 13z — 34 has roots z = 2, z = —1 + 4i, and 
z = —1 — 4i. The first root is real, the latter two are complex-conjugate. 


Exponentiation and Euler’s Identity 


Now that we’ve defined the basic four arithmetic operations for complex numbers, 
let’s move on to the transcendental functions—exponentials, logs, trig functions, etc. 
In particular, how should we define exponentiation? This is the key to much of what 
follows in this text. 

Our initial goal is to make sense of the expression e? where z = x + iy is a complex 
number in rectangular form. If we require that the familiar rules for exponentiation 
hold (in particular, e°? = ee”) then we need 


E = et — eel, (A.16) 


The quantity e* is just the usual exponential of a real-valued quantity x. But we need 
to make sense of e”, the exponential of a pure imaginary number. 
Let’s start by defining a function f(y) as 


fly) =e”. (A.17) 


It’s clear that e” is itself likely to be a complex-valued quantity and so has a real part 
that depends on y and an imaginary part that depends on y. This means f(y) can also 
be written as 


f(y) = gy) + ih(y) (A.18) 


A.1 Complex Numbers and Arithmetic 


where g(y) is the real part of f(y) and h(y) is the imaginary part. Here g and h are 
ordinary real-valued functions of a real variable. 

Let’s assume the familiar rules for differentiation hold for complex-valued func- 
tions. If we differentiate f as given by equation (A.17) with respect to y we obtain 


f(y) = ie” 


= —h(y) + ig(y) (A.19) 


where we used (A.18). Similarly if we differentiate f as given by equation (A.18) with 
respect to y we obtain 
f(y) =9'(y) + ih'(y). (A.20) 


Compare equations (A.19) and (A.20); the right side of each is an expression for f (y), 
and so by matching real and imaginary parts we conclude that the functions g(y) and 
h(y) must satisfy the equations 


g'(y) = —h(y) (A.21) 
h'(y) = gly). (A.22) 


From (A.18) we also see that f(0) = g(0) + ih(0). If we want to e° = 1 to still hold we 
need g(0) = 1 and h(0) = 0. 

Equations (A.21)-(A.22) are a pair of coupled first order differential equations. With 
the requirement that g(0) = 1 and h(0) = 0 you can easily verify that g(y) = cos(y) 
and h(y) = sin(y) satisfy (A.21)-(A.22) as well as the conditions at y = O. (In fact, as 
you’ll see later this is the only choice for g and h). Accordingly, from (A.17)-(A.18) we 
conclude that if y is any real number then e” = cos(y) + /sin(y). In fact, we will take 
this as a definition: 


Definition 43 Euler’s Identity For any real number y we define e” as 


e” = cos(y) + i sin(y). (A.23) 


Equation (A.23) is known as Euler’s Identity. It’s the key to understanding how to ex- 
ponentiate complex numbers, as well as take logarithms, roots, sines, cosines, etc. 


Example 132 Compute e’” 


SOLUTION From (A.23) with y = 7 we find (since cos(7) = —1 and sin(7) = 
0) . 
e" =—1. 


Math fans write this as e + 1 = 0 and print it on t-shirts, since it embodies five of the 
most fundamental numbers in mathematics in one equation. 
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Example 133 Compute e2”" where m is an integer. 
SOLUTION For any integer m we have, from (A.23) with y = 27m, 
e?™™ — cos(2mm) + isin(2am) = 1 (A.24) 


since cos(27m) = 1 and sin(27m) = 0. 


To exponentiate a general complex number z = x + iy we’ll use (A.16). 


Definition 44 The Exponential Function For any complex number z = x + 
iy we define 
e = e“cos(y) + ie* sin(y). (A.25) 
Example 134 Compute e2+?, 
SOLUTION We find 


e? +3! — e? cos(3) + ie? sin(3) ~ —7.315 + 1.043). 


Exercises A.1 


1. Find the real and imaginary part of (f 
each complex number below, and state (g 
whether the number is real, purely 
imaginary, or neither. 


) u/(wz) 

) u(w + z) and uw + uz 

(h) 1/z 

(i) 2 
) 


Z 


(a) 2 — 3i ih v=. 

(b) etmi 3. Compute each of 7,/7,7,7,i°,/’,#, 
(c) 5i and see the pattern. Then compute }?!° 
(d) -7 — 5i without using any computational aid. 


. Letu = 2 + 3i,w = 1 — i, andz = 
2 + i. Compute and simplify to rectan- 
gular form each of 


(a) w+z 
(b) 3w—z 
(c) wz 


(d) (uw)z and u(wz) (associate each 
way) 


) u/z 


a For 21 = 4 + Gi, z2 


. fz = 2+ 3iandw = 1- i, compute 


each of Z, w, Zw, and z/w. 


. For the complex numbers zı = 4 — 3i, 
2= —8 + 3i, Z3 = 4, Z4 = 5 + 6iand 
Zs = 7. Compute: 


(a) 21 + 22,21 + 22,21 +22. 
(b) Z4 — 3z, 4Zs = 321 + 273. 


(c) Zs 221 = 423. 


= 9 — 3i, Zz = 
—3 + 11i Compute: 


A.2 The Geometry of Complex Numbers 


(a) 2123. 10. A fifth degree polynomial p with real 

(b) z223. coefficients has roots —3, —2 + i, and 
—7 — 5i. What are the other roots of 

(c) Z1(Z2 + 523). p? 

(d) 212223. 


(e) 2121, Z272, and z373. What pattern 
do you notice? 


7. With z1,Z2,Z3 as in the last problem, 


write the following in the form a + ib: 


11. 


12. 


Carry out the algebra necessary to solve 
equations (A.7) and justify the defini- 
tion (A.8). 


Compute y3 + 4i, that is, find a com- 
plex number(s) z that satisfies z? = 3+ 


(a) z1/zs. 4i. Hint: Let z = a + bi, expand z* out 
(b) 22/23. in terms of a and b, set the result equal 

to 3+ 4i (real and imaginary parts), and 
(c) z1/ (22/23). solve for real values a and b (two equa- 
(d) —- tions, two unknowns). 


Z12223 ` 

. Use the quadratic formula to compute 
both roots of 2z* — 4z + 4 = 0. Verify 
that the roots are in fact complex con- 
jugates of each other. 


. Use the quadratic formula to directly 
verify that the solutions to ax” + bx + 
c = Oare either both real or complex- 
conjugate to each other. 


13. 


14. 


Use Maple to numerically compute all 
roots of Z° — 3z? + 3 = O. Verify that 
all roots are either real or come in con- 
jugate pairs. 


Use Maple to numerically compute all 
roots of Z° — 3z*+3i = 0. Are the roots 
allin conjugate pairs? Why doesn’t this 
violate equation (A.15)? 


A.2 The Geometry of Complex Numbers 


AS YOU READ 


1. What is the polar form of a complex number, and what is the polar form good 
for? 


2. How do we interpret complex multiplication and division geometrically? 
3. T/F: The modulus of a complex number is always nonnegative. 
4. How are |z”°| and |z|? related? 


5. T/F: The principal argument of a real number is 0. 


The Complex Plane 
Visualizing Complex Numbers 


Visualizing the real numbers as a line brings the power of geometric insight to the 
mathematics. Visualizing the complex numbers as points or vectors in the plane does 
the same. In Figure A.1 the complex number z = x + iy (rectangular form) is shown 
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on a pair of cartesian axes, both as the point (x, y) and as a displacement vector (x, y) 
from the origin. Which way you visualize it depends on what you're trying to do, and 
personal tastes. Complex numbers that lie on the horizontal axis are of the form z = 


z=x+iy z=x+iy 

Y- Y- © 
N N 
2 2 
< < 
aa pan 
© © 
C (= 
5 5 
© © 
£ £ 

| | 

Real Axis X Real Axis X 


Figure A.1: Complex number z = x + iy depicted as a vector (left) and point (right). 


x+0iwhere x is real, hence the horizontal axis is called the real axis when we’re plotting 
complex numbers. Similarly the vertical axis is the imaginary axis. 


Modulus and Argument 


As we’ll soon see, it can be extremely useful to describe complex numbers in polar 
coordinates. Recall that a point (x, y) in cartesian coordinates has polar coordinates 
rand 0, where the quantity r is the distance from to origin to the point (x, y) and 0 
is the angle that a line joining the origin to (x, y) makes with the horizontal axis. By 
convention we take —7 < @ < vr, though it is sometimes useful to move outside this 
range. 

It’s easy to compute that r = ,/x2 + y2, by drawing a right triangle with horizontal 
leg of length x, vertical leg of length y, and hypotenuse r. We also see that tan(@) = 
y/x. However, the formula 0 = arctan(y/x) is correct only if x > 0 (so —7/2 < 0 < 
m/2). If (x,y) isin the second quadrant we need to add z to arctan(y/x), while if (x, y) 
is in the third quadrant we have to subtract 7 from arctan(y/x). All of this is embodied 
in the now “standard” arctangent function arctan(y, x) defined as 


arctan(y/x), x>0 
arctan(y/x) +7, x<Oandy>0 
arctan(y,x) = 4 arctan(y/x)-—7, x<Oandy<0 (A.26) 
m/2, x =Oandy>0 
—1/2, x =Oandy <0 


that accepts two arguments (the coordinates of the point of interest, in the order y, x) 
and returns the polar angle @ for this point in the range —7 < 0 < m. This notation 
and command is built into most modern programming languages and software. The 
quantity arctan (0, 0) is undefined. 


A.2 The Geometry of Complex Numbers 


When working a complex number = x + iy, we call the quantity \/x* + y? the 
modulus of z and write |z| = ./x* + y*. The polar angle 6 is called the argument of z 
and is notated arg(z). In summary, if z = x + iy 


Definition 45 Modulus and Argument If z = x + iy is a complex number in 
rectangular form we define the modulus |z| and argument 
arg(z) as 


lz) = f/x? + y? (A.27) 


arg(z) = arctan(y, x). (A.28) 


In light of the fact that there can be some ambiguity in the “angle” arg(z), some texts 
refer to this quantity as the principal value of the argument. 
The geometry is shown in Figure A.2. 


Imaginary Axis 


|z| sin(arg(z)) 


arg(z) 


x = |z| cos(arg(z)) ———— 


Real Axis 


Figure A.2: Modulus |z| and argument arg(z) of complex number z. 


Example 135 Let z = 1 + i. Compute |z| and arg(z). 
SOLUTION We find |z| = v12 + 12 = V2 and arg(z) = arctan(1,1) = 7/4. 


Of course |z| > 0, and |z| = 0 if and only if z = 0. The modulus of a complex num- 


259 


Chapter A Complex Numbers 


260 


ber is an extension of the notion of “absolute value” for real numbers. The modulus 
may in fact be referred to by some authors as the “absolute value” or “magnitude” 
of z. As in polar coordinates, it is conventional to take the argument in the range 
—m < arg(z) < m. Then arg(z) = 0 when z is a positive real number and arg(z) = 7 
when zis negative real. The argument of 0 is undefined. 


Polar Form 


From the rectangular-polar relations x = rcos(@) and y = rsin(@) we see that for a 
complex number z = x + iy we have Re(z) = |z| cos(@) and Im(z) = |z| sin(@) where 
0 = arg(z). As a result we can express z as 


z = |z| cos(@) + ilz| sin(6). (A.29) 


Equation (A.29) actually makes sense for any real 6, whether or not —7 < 6 < v, and 
it’s sometimes useful to allow that. 

Based on Euler’s Identity (A.23) and (A.29) we see in fact that any complex number 
z can be expressed as z = |z|e” with 0 = arg(z). This is called the polar form of the 
complex number z. 


Definition 46 Polar Form For a complex number z we have 


z= |zļe "©, (A.30) 


The right side of (A.30) is called the polar form for z. 


We often use f in place of arg(z) for convenience. 


Example 136 Convert z = —2 + 2i to polar form. 
SOLUTION We find that |z| = 2\/2 and 6 = arctan(2, —2) = 37/4, so that 
z= 2V2e8"/4., 


Remark 1. Equation (4.30) gives one way to write z in polar form, but it’s worth noting 
that we can always add or subtract an integer multiple of 27 to arg(z) in (A.30) and 
the equation remains true, since for any integer m 


jiarg(z)-+2mir iarg(z) emin 


izje = |zle 
= |zļe 
=z 


(recall Example 133). 


A.2 The Geometry of Complex Numbers 


The Geometry of Arithmetic Operations 
The Geometry of Addition and Subtraction 


A complex number z = a + bi can be identified with a vector (a,b). Given that com- 
plex addition and subtraction are algebraically identical to their two-dimensional vec- 
tor counterparts, we can visualize addition and subtraction of complex numbers in a 
fashion similar to that of vectors in two-dimensions, by thinking of the complex num- 
bers as vector displacements from the origin as in the left panel of Figure A.1. 


The Geometry of Multiplication 


The polar form yields fundamental insight into the geometry of complex multiplication. 
Consider two complex numbers z; = x1 + jy; and z2 = X + iy in rectangular form. 
Using (A.30) we can express 


zı = zie 
Z: = \z2 |e" 
where 6; = arg(zı) and 02 = arg(z2). The product z = 212, is given by 

Z = 2122 = [21 ||z2|e"6!% = \21z2| e014) (A.31) 


From the right side of (A.31) (and recall Remark 1) we deduce that |z| = |z,||z2| and 
arg(z) = arg(z1) + arg(z2). This leads to an important result: 


Theorem 25 Modulus and Argument of a Product If z4 and z2 are any 
complex numbers then 


|z1z2| — |zal|z2] (A.32) 


arg(zız2) = arg(zı) + arg(z2) (A.33) 


if the sum on the right in (A.33) is reduced to the range 
(=r, 7]. 


Thus for complex multiplication the modulus of the product is the product of the mod- 
uli and the argument of the product is the sum of the arguments. This vital geometric 
conclusion is illustrated in Figure A.3. 


Example 137 Letz = 1+iand z) = —3i. Compute the moduli and arguments 
of Z1, Z2, and Z122. 


SOLUTION We find |z1| = V2, |z2| = 3, arg(z1) = 7/4, and arg(z2) = —7 /2. 


According to (A.32)-(A.33), |21Z2| = 3/2 and arg(zız2) = 1/4 — 1/2 = —7 /4. This is 
easily seen to be correct by directly computing z1z2 = 3 — 3i. 
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Figure A.3: For complex multiplication moduli multiply, arguments add. 


Repeated application of (A.32)-(A.33) shows that for any integer n > 0 


(A.34) 


\z"| = |z|" 
(A.35) 


arg(z”) = n arg(z). 


Example 138 Let z = 1 + i. Compute z® by using (A.34) and (A.35). 

SOLUTION We find |z| = v2 and arg(z) = 7/4. From (A.34) we have |z°| = 
\z|° = (21/2)© = 23 = 8 and arg(z®) = 6(7/4) = 37/2. However, we can reduce 
37/2 to the conventional range —7 < 0 < 7 by subtracting 27, so that arg(z°) = 
37/2 — 2r = —7/2. This means that zê lies on the negative imaginary axis at a dis- 
tance of 8 from the origin, so zê = —8i. This can be verified by computing (1+ iP 


directly. 


Example 139 If z? = 1, what are the possible values for z € C? 


SOLUTION Suppose z = r(cos(@) + isin(@)). From (A.32)-(A.33) we see that 
|z| = |z|? = r? and arg(z*) = 2 arg(z) = 20. If z? = 1 (note |1| = 1 and arg(1) = 0) 
then it would seem that we need r? = 1 and 20 = 0, so (since r > 0)r = 1and 0 = 0, 
corresponding to z = 1. But there is another possibility: if we take z = r(cos(0) + 


A.2 The Geometry of Complex Numbers 


isin(@)) with r = 1 and 0 = 7 then 
z? = 1*(cos(2m) +isin(27)) = 1. 


Of course r = 1 and 0 = ~ corresponds to z = 1(cos(m) + isin(z)) = —1, so 
(—1)? = 1 too. 


The Geometry of Division 


To examine the geometric interpretation of complex division, suppose that w = z;/Zp. 
This is equivalent to zı = wz. From (A.32) we conclude that |z;| = |wz2| = |w||z2| so 
that |w| = |z,|/|z2|, or 


|z1/z2| = |Z1|/|Z2|- (A.36) 


The modulus of the quotient is the quotient of the moduli. Similarly from z4 = wz, we 
obtain from (A.33) that arg(z,) = arg(w) +-arg(z2) so that arg(w) = arg(z,) — arg(z2). 
Thus 


arg(z1/z2) = arg(z1) — arg(z2). (A.37) 


The argument of the quotient is the difference of the arguments. 


Exercises A.2 


1. Plot each complex number z below ona 
pair of real/imaginary axes, along with 
Z. In each case find the modulus and ar- 
gument of the z, either by using (A.27)- 
(A.28), or by visual inspection. Find the 
modulus and argument of Z. How are 
|z| and |Z| related? How are arg(z) and 
arg(z) related? 


(d) |z| =5,arg(z) = —7/2 


3. For the complex numbers z = 2 — i and 
w=5+4i. 


(a) Draw the vectors in the complex 
plane associated with the com- 
plex numbers z and w. 


(b) Compute the complex number 


(a) z=1+i z + w, and draw the vector asso- 

(b) z=7 ciated with z + w. How does this 
vector compare with the vectors 

(c) z = —3 + 3i for z and w? 

(9) z= =s (c) Compute the complex number 

(e) z=i z — w, and draw the vector asso- 

(f) z=1+iv3 ciated with z — w. How does this 


vector compare with the vectors 
2. For each modulus |z| and argument for z and w? 
arg(z) below, find the corresponding 
complex number z = x + iy and plot 


it on a pair of axes. 


4. To help visualize complex numbers, for 
the complex number z = x + iy, we as- 
sociate z with the vector V = (x,y). 


(a) Draw the vector in the xy-plane 


associated with the complex 
number z = 3 + 4i. 
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(b) Draw a vertical line from the tip 
of the vector to the x-axis. What 
is the length of this line? What 
is the distance from the origin to 
the bottom of the line? Label 
these lengths. 


(c) In your picture, label lengths that 
represent the real part of z, the 
imaginary part of z and the mod- 
ulus of z. 


(d) Draw the vector in the xy-plane 
associated with Z. What is hap- 
pening to the vector z to get Z? 


(e) Explain why it makes sense to call 
the y-axis the “imaginary axis” 
and the x-axis the “real axis.” 


(f) Was there anything special about 
the complex number z = 3 + 
4i? How would the previous parts 
change if we instead used the 
complex number w = —2 + 5)? 


. Express each of the following in polar 
form re” by finding r and 6, with @ in 
the range -r <0< 7. 


(a) 3 

(b) 1+i 
(c) —2 + 2i 
(d) i 

(e) —7 

(f) 5e". 


. Use the polar form of each complex 


number z in the last problem to com- 
pute z° in rectangular coordinates. 


. Show that zZ = |z|?. Hint: just suppose 


z = x + iy, and compute both sides. 


. Letz = —2 + 2i and w = 3i; compute 


|z|, |w|,arg(z), and arg(w). Then use 
this along with (A.32)-(A.33) or (A.36)- 
(A.37) to compute the modulus and ar- 
gument of each of the following quanti- 
ties. Reduce the argument to the range 
(—7, 7] by adding or subtracting a suit- 
able multiple of 27. Finally, compute 
each quantity directly (use Maple when 
appropriate) and verify your answers. 


(a) z? 
(b) wz 
(c) wz? 
(d) w/z 
(e) w/z? 


(f) w/z 


. Suppose z = cos(27/3) + isin(27/3). 


Use (A.34)-(A.35) to show that z? = 1 
(so z is a cube root of 1). Repeat for 
z = cos(47/3) + isin(47/3). Finally, 
show that there is a third cube root for 
1. Hint: it’s real! These three numbers 
are called the cube roots of unity. 


. Find all complex numbers z such that 


z’ = 1 (the fourth roots of unity). 


A.3 Transcendental Functions of a Complex Variable 


AS YOU READ 


1 


2 


. T/F: A typical complex number z has n distinct “nth roots.” 


. What are the nth roots of unity? 


Do negative real numbers have meaningful logarithms? 


. Can we take the sine of any complex number? 


A.3 Transcendental Functions of a Complex Variable 


In this section we’ll explore some consequences of Euler’s Identity (A.23) a bit 
more, arithmetic using the polar form (A.30), and how to compute a variety of other 
transcendental functions (roots, logs, trig) of a general complex number. 


More on Polar Form 
Arithmetic with the Polar Form 


By taking advantage of Euler’s identity and the algebra of exponentials, we can deduce 
some simple rules for multiplication, division and exponentiation of complex numbers. 
Example 140 Let z, = 1+iandz) = —1 + i. Convert each to polar form, as well 
as Z1Z2. Sketch each in the complex plane. 


SOLUTION If z, = 1 +iandz, = —1 + i, then |z;| = v2 and |z,| = v2. We 
find 6; = arg zı = 7/4 so that the polar form is 


z =ne = /2e"/4, 
We find 0, = arg z = 3iz/4 so that the polar form is 
z = ne = 2e, 


A sketch is given in Figure A.4. 
Note that we can verify these polar forms by using Euler’s formula, for example 


V2e'"/* = /2[ cos(r/4) + isin(m/4)] -va +2] Sda 


To best utilize this exponential-based polar form, we recall the following properties 

of the algebra of exponentials. 
; ; eft : ay k $ 
elt e2 = e!(01+02) | = e!l), (e) = e? 
el2 

Here k can be any number. If z; = ne™, z = ne”. 

Given two complex numbers z; = re and z2 = re”, complex multiplication in 
polar form is defined by 
iða „i02 


zz = ne" re! = nnee 


using properties of multiplying exponentials, we see 


zz = nnee = rr elt’), 


Similarly, we can utilize the algebra of exponentials to quickly compute division of com- 
plex numbers in polar form 
iO, iO, 


Z1 re n e 


r1 i(01—62) 
ið. iĝ e ` 
Z2 mne”? n e2 h 
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Imaginary Axis 


Real Axis 


Figure A.4: Complex numbers 23, Z2, and z;z2 on the circle of radius |z| = 2. 


When computing powers of complex numbers, the polar form can be quite useful. 


Noting that in many applications, it is useful to return these complex numbers to the 
rectangular x + iy form, we summarize these properties as follows. 
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Definition 47 Multiplication and Division with Polar Form Given two 
complex numbers z; = re and z2 = ne”, 
Zz = nnee = rr cos(O1 + 02) + irra sin(O1 + 02). 


Division of complex numbers yields 


Z1 i Hh ri E 
2 = telli—%) — = cos(01 — 02) +i— sin(04 — 02). 
Z2 h2 r2 r2 


While exponentiation yields 


zk = kel = rk cos(k0,) + irk sin(k01). 


Example 141 Let zı = 1 +i and z2 = —1 + i. Convert each to polar form and 
compute 2122, Z1 /Z2 and z}°. 


SOLUTION We saw in Example 140 that 
z= V2e7/4, and z = V2e27/4. 


Thus, 
ZZ = (V2e'"/) (v2e3"/*) = Qel(a/4+37/4) = 2e%7/4 = 2e". 


In rectangular form, 
ZZ) = 2e = 2cos(m) + i2 sin(7) = —2. 


Now, ; 
Z1 V2ei7/4 = Yei(n/4—-31/4) — 9—in/2 


ran V2e3"/4 
In rectangular form, 


a =e '"/2 — cos(—r/2) + isin(—r/2) = ~i. 
2 


Finally, 
zi0 = (V2e'7/*) 10 _- (/2)elon/4 — 210/2,/107/4 _ 39 ,i107/4 
In rectangular form, 
z£? = 32e'10"/4 — 32 cos(10m/4) + i32 sin(107/4) = 32i. 


We can verify all of the results above using the rectangular form. However, computing 
re by multiplying z4 by itself ten times will be time consuming. 
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Imaginary Axis 


Figure A.5: Complex numbers z4 and Z3. 


Complex conjugation also works well in the polar form. Since ris real, we have 


Z= re = re", 


Remark 2. This section showed how the polar form, along with the algebra of expo- 
nentials can be used to streamline computations for multiplication, division and tak- 
ing powers of complex numbers. There are a few things we should notice. First, the 
polar form is not useful when doing addition or subtraction. To compute zı + z), we 
should use the rectangular form. Second, for multiplication and division, the polar form 
will only simplify computations if we already have the moduli r1, r) and the arguments 
01, 92. Otherwise, it is usually simpler to use the rectangular form. Finally, to compute 
the arguments 6 well usually requires the values to be familiar angles on the unit circle 


N T TT T 
suchas 0, $, Z 32 


Logarithms and Roots 
Roots 


Let n be a positive integer and z € C; we'll assume z Æ 0, so |z| > 0. We're interested 
in computing all solutions to w” = z, the “nth roots of z.” We use polar form z = Izle 


A.3 Transcendental Functions of a Complex Variable 


and w = |wļ|e! where 0 = arg(z),a = arg(w). Then w” = z becomes |w|"el"® = 
|z\e"’. An obvious matching of moduli and arguments gives |w|" = |z| and na = 0 
from which we conclude that |w| = |z|/” (the usual nth root of a nonnegative real 
number) and a = @/n, so that w = |z|!/"e®/". But it turns out this is not the only 
choice for w. 

Instead of writing z = |z|e” for z, let’s use the fact that z = |z|e”+?" is also valid 
for any integer k (look back at Example 133). Then similar reasoning as in the last 
paragraph, starting with w” = z, yields 


|w" e". = eee 
= |z|e!(0 +27) 
Matching moduli again produces |w|" = |z|, so |w| = |z|'/" as before. Matching 


arguments above gives na = 0 + 2km, so that 
a = 0/n+ 2kr/n. (A.38) 
Then solutions w to w” = zare of the form 


w= |z|1/"e!®/n+2kr/n) 
= (ei) e2kni/n (A.39) 


Different choices for k give different solutions w to w” = z. 

But there are only a finite number of k that yield different solutions. The parenthe- 
sized expression in (A.39) doesn’t depend on k, but as k ranges from k = Otok = n—1 
the quantity e2kni/n generates n distinct complex values, given by 


e™"i/n = cos(2kr /n) + isin(2k7/n). (A.40) 


We then obtain n distinct solutions to w” = z from (A.39). Nothing is new is obtained 
by taking k outside the range 0 < k < n—1: if two integers k and k’ differ by a multiple 
of n, say k' = k + mn for some integer m, then 


jae > ; i : 
2k'ri/n __ e2(k+mn)nri/n 2kni/n 2m7i — e2kni/n 


e =e 


since e*™™ = 1 for any integer m. Thus k and k’ in (A.40) or (A.39) yield the same value 
for e2k//" or w. The range 0 < k < n — 1 is all we need. 


Key Idea 18 Complex nth Roots Any nonzero complex number z has n 
distinct “nt” roots,” that is, the equation w” = z has n dis- 
tinct solutions, given by (A.39). The n numbers generated by 
(A.40) are these roots in the special case that z = 1 (so we 


use |z| = 1,4 = 0 in (A.39)) and are called the nth roots of 
unity. They all have modulus 1 and lie at equispaced angles 
on the unit circle |z| = 1. 
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Example 142 Compute e2*7/3 


circle |z| = 1 in the complex plane. 


for each of k = 0,1, 2. Plot these numbers on the 


SOLUTION These are the third roots of unity and we find 


. ; 1 1 : 1 #1 
zı eù l, z e27!/3 = + 5iv3, z= efti/3 E = = 5iv3. 


Where we used Euler’s formula to convert to rectangular form. A plot is shown in Fig- 


ure A.6. You can verify directly that z? = z = z3 = 1. 


Imaginary Axis 


01 Real Axis 


Figure A.6: The third roots of unity on the unit circle |z| = 1. 


The use of the polar form is for finding the roots of polynomials of the form f(z) = 
z” — w for some integer m and complex number w. 


Example 143 Find all solutions of the equation z? + 8 = 0. 


SOLUTION We note that this is equivalent to finding all complex numbers z 
so that z = —8. We begin by writing —8 in polar form, 


8 = Belt terik k =0,+1,+2,... 
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Then, since | — 8|1/3 = 2, we have 
(—8)*/3 = 3-1/3 (arene) 1/3 _ eit /3+2nik/3 
We select k = 0,1, 2 to find our three solutions: 
k=0: 2e/3 = 2cos(x/3) + i2sin(/3) = 1 + iv3 
k=1: 2e'7/3+27i/3 — 2 cos() + i2 sin(7) = —2 
k=2: 2e!7/3+47//3 — 9 cos(57/3) + i2 sin(57/3) = 1 —iV3 


Note that if we chose k = 3, 
k=3: 2e!7/3+27! — pelt /3e27! — Jeit/34271 — 2 cos(r/3) + i2 sin(x/3) = 1 + iv3 


returns the same value as k = 0. These results are plotted in Figure A.7. 


1+ i/3 


Imaginary Axis 


Real Axis 


1—i/3 


Figure A.7: The third roots of —8 on the circle of radius 2. 


This strategy may be employed to find the roots of any complex number. 


Example 144 Find all solutions to w = 2i (we might reasonably called these the 
“Ath roots of 2i”). 
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SOLUTION 


First, |2i| = 2 and arg(/) = 7/2. From (A.39) we find roots 


Pal 21/4 ei(n/8+1/2) oie ase 21/4 oi(a/8+37/2) 


which are approximately 1.099 + 0.455/, —0.455 + 1.099/, —1.099 — 0.455/, 0.455 — 


1.099/. The results are plotted in Figure A.8. 


—0.455 + 1.099; 


-1.099 + 0.455/ 


b= 1/8 


—1.099 — 0.455/ 


Real Axis 


0.455 — 1.099; 


Figure A.8: The solutions to w = 2i ona circle of radius 22/4, 


Notice that while the complex roots of a real number come in conjugate pairs, they 
do not for numbers with a non-zero imaginary part. The various roots do continue to 
be equally spaced 27 /n radians apart on a circle in the complex plane. 


Logarithms 


Since we can exponentiate complex numbers, it stands to reason we should be able to 
take logarithms. To say that w = In(z) means that z = e". Suppose that w = a + bi 
and z = x + iy; we can solve for a and b in terms of x and y by noting that if z = e” 


then 
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x + iy = e™* = e" cos(b) + ie’ sin(b) 


(A.41) 


A.3 Transcendental Functions of a Complex Variable 


if we make use of (A.25). Equating the real and imaginary parts on both sides of (A.41) 
shows that we need a and b to satisfy 


e° cos(b) = x and e° sin(b) = y. (A.42) 


To solve for a, square both sides of the equations above and add to obtain e?” = 
,/x2 + y?. Since a is assumed real, we can take an ordinary natural logarithm and do 
a bit of algebra to find 


a= 5 ING? + ¥2) = In( yx? + y?) = In |z]. (A.43) 


To find b, exponentiate both sides of (A.43) to see that e° = |z|, so (A.42) can be 
written 


cos(b) = x/|z| and sin(b) = y/|z|. (A.44) 


In vector terms, (A.44) can be expressed as 
(cos(b), sin(b)) = (x/|z|,y/|z|). (A.45) 


Both sides of (A.45) are unit vectors; all we need is an appropriate angle b. One obvious 
choice is b = arctan(y/|z|,x/|z|) = arctan(y,x) = arg(z) but in fact, we can add any 
integer multiple of 27 to this choice and (A.45) remains true. That is, we can take 


b = arg(z) + 2kr (A.46) 


for any integer k. Since In(z) = w = a + bi we have shown that there are infinitely 
many solutions to e” = z, all of the form 


w = In |z| +i arg(z) + 2kzi (A.47) 


where k is an arbitrary integer. Note that In |z| on the right in (A.47) is the usual loga- 
rithm of a positive real number. 

Any of the choices for k in (A.47) yields a solution to e" = z, so any of these distinct 
w could be termed the “logarithm of z.” But the case k = O guarantees that arg(w) 
lies in the range —7z to 7, and this is the conventional choice. 


Definition 48 Principal Branch of the Logarithm For a nonzero complex 
number z we set 


In(z) = In |z| + i arg(z). (A.48) 


This is called the principal branch of the logarithm. 


Example 145 Compute In(1 + i). 
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SOLUTION If z = 1 + i then |z| = V2 and arg(z) = 7/4, so 
In(1 + i) = In(V/2) + in /4. 


Example 146 Compute (1 + imt. 


SOLUTION We can compute z” for any complex z and w, as follows. First, 
write z = e!") (we'll stick with the principal branch of the logarithm), so that z” = 
(el"@))” = enG) if we're willing to believe the usual rules for exponentials work. 
Thus for example, let’s compute (1 + /)~*t! (soz = 1+i,w = —1 + i and In(z) = 
In(/2) + i7 /4.) Then 


(1 +i) Iti — p(—1+/)(In(v2)+in/4) 


— e(—In(V2)—1/4)-+i(In(V2)—7/4) 
= el "(V2)—7/4) (cgs(In(/2) — 1/4) + icos(In( v2) — 7/4)) 
= 0.292 — 0.137/. 


Note that in the second-to-last line the exponentials and trig functions are applied to 
real numbers. 


Sine and Cosine 
We can substitute 0 = —0 into Euler’s Identity e? = cos(0) + i sin(0) and find 
e’ = cos(—0) + isin(—0) 
= cos(@) — isin(0) (A.49) 


since the sine function is odd and cosine is even. If we add Euler’s Identity (A.23) and 
(A.49) and divide both sides by 2 we obtain 


ef 4 ei 
2 


Thus the cosine function can be expressed as a linear combination of complex expo- 
nentials! Similarly, subtracting (A.49) from (A.23) and dividing by 2i yields 


cos(@) = 


el? 


sin(@) = —— 


2i 
The equations hold for any real 8. 

But we can use these to extend the definition of sine and cosine to all of the com- 
plex plane, by simply defining 


eZ e 
cos(z) = = (A.50) 
IZ e 2 
sin(z) = A.51 
in(z) zi (A.51) 
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for any complex number z, since we know how to compute e? and e~”. 


Example 147 Compute cos(1 + i). 


SOLUTION We find 


eitti) 4 eili) 
2 
enti p el-i 
2 
ete! +ee! 
2 
e~*(cos(1) + isin(1)) + e(cos(1) — i sin(1)) 
2 
(e~1 + e) cos(1) K Gik — e) sin(1) 
2 2 
zæ 0.834 — 0.989i. 


cos(1 +i) 


Of course we can then define tan(z) = sin(z)/ cos(z) = e" te as well as the var- 


ezZ—e—z 


ious other trigonometric and inverse trigonometric functions. 


Exercises A.3 


1. For each complex number z below, (f) (=7 —247)"" 
compute the nth roots of z for the given 


3. Recall the algebraic identit 
n. Then plot the roots in the complex j z 


plane. 1—z" = (1—z)(14+-z+274+---+277") 
(a) z = 1,n = 4 (the 4th roots of Use this to show that z = e%™'/” satis- 
unity). fies the equation (1 +z +7 +- + 


(b) z=2+2in=3 z"™+) = 0 if k is not a multiple of n. 


(c) z=—1,n=4 4. Compute the roots of z* + z? + 
(d) z= 16,n = 4. 1 = 0 and plot them on a pair of 
real/imaginary axes. Hint: Start with a 


. In each case below, find all of the indi- 
cated roots. In each case there are 2, 3, 
or 4 solutions! 


substitution u = 7°. 


. Compute or approximate the real and 


imaginary parts of the natural loga- 
rithm of each number below, as explic- 
itly as possible. 


(a) 3 
(b) cos(1) + isin(1). 
(c) ecos(1) — ie sin (1). 
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(d) i (A.30). 
= . Use (A.48) to compute In (iy) where y > 
(e) —7 7. Use (A.48) In(iy) wh 
(f) 5e". O is real (the answer involves In |y|). 
6. Exponentiate both sides of (A.48) 8. Compute i’. Use the principal branch of 
and verify it degenerates to equation the logarithm. 


A.4 More on Functions of a Complex Variable and Calcu- 
lus 


AS YOU READ 


1. How many distinct complex roots can an nth degree polynomial have? 
2. If ris a root of a polynomial p(x), what is the “multiplicity’ of r? 


3. T/F: The usual rules for differentiation (produce, quotient, chain, etc.) apply to 
complex-valued functions. 


Polynomials and the Fundamental Theorem of Algebra 
Polynomials and Roots 
An nth degree polynomial p(z) over the complex numbers is a function 
p(z) = ao + qız + a2z? + +++ + anz” (A.52) 
in which the a, € C and a, Æ 0. You’ve encountered plenty of first degree (linear) 


polynomials p(z) = ao + a1z and quadratic polynomials p(z) = ao +aız + a27? before. 
A complex number a is a root of p if p(a) = 0. A first degree polynomial has exactly 


one root, Z = —@o /aı. A quadratic typically has two roots, 
—a, + fat — 4aga 
nS 1+ 1 0492 
2a2 i 
though these roots coincide if a? — 4a9az = 0. In this case the root z = —a,/(2a2) ia 


a root of multiplicity 2, or a double root. 


The Fundamental Theorem of Algebra 


Maybe you’ve been told, or just noticed, that a 3rd degree (cubic) polynomial usually 
has three roots, a 4th degree (quartic) has four roots, and so on. These are special 
cases of the Fundamental Theorem of Algebra, which comes in various forms. For our 
purposes the theorem can be stated as 


A.4 More on Functions of a Complex Variable and Calculus 


Theorem 26 Fundamental Theorem of Algebra Let p(z) = ao + aiz + 
--- Anz” be an nth degree polynomial, n > 1, with coeffi- 
cients a, € C (anda, # 0). Then p(z) has exactly n com- 
plex roots Q1, Q2,°-- , Œn, not necessarily distinct, in C. In 
this case p(z) can be factored as 


p(z) = (z—a1)(z — a2) +++ (Z — an). (A.53) 


Example 148 Factor the polynomial p(z) = 2 —2(1+i)z?+(—2+3i)z+(3—i) 


SOLUTION The roots are best found via Maple and are z = 1,z = —1 + i, and 
z = 2 + i. Thus p(z) can be factored as 


p(z) = (z — 1)(z — (—1 + i))(z — (2 + i)). (A.54) 


Although the Fundamental Theorem 26 isn’t usually central to any particular com- 
putation we do, it provides an important theoretical and conceptual framework. 

As stated in Theorem 26, the roots a, are not necessarily distinct; a given root may 
appear multiple times. Let @ for 1 < k < r be the number of distinct roots for p, and 
m; the number of times 4 appears as a root of p in (A.54). Then (A.54) can be written 
as 


p(z) = (2— A)™ (z - B)™---(@- A)” (A.55) 


where ris the number of distinct roots. The integer m, is called the multiplicity of the 
root 3, and then m, +m +:::m, =n. 


Definition 49 Multiplicity of a Root If a polynomial p(z) factors as in (A.55) 


then mx is called the multiplicity of the root ĝ. 


Example 149 Factor p(z) = zê — 72° + 17 zf — 13 z? — 10z* + 20z— 8 and find 
the multiplicity of each roots. 


SOLUTION Again, this is best done via Maple. The factorization is 
p(z) = (z — 1} (z + 1)(z — 2}. (A.56) 


Thus p has three distinct roots: 1 is a root of multiplicity 2, —1 is a root of multiplicity 
1, and 2 is a root of multiplicity 3. Note that the sum of the multiplicities adds up to 
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the degree of p,i.e.,2+1+3 =6. 


The Fundamental Theorem was believed to be true, in some form, as early as the 
17th century. Many first-rate mathematicians offered proofs that turned out to be er- 
roneous or incomplete. It wasn’t until the early 19th century that a rigorous proof was 
offered (usually credited to Gauss in his PhD thesis), but even this proof had shortcom- 
ings that weren’t entirely resolved until the 20th century! 

Take note, however, that just because an nth degree polynomial must have n roots 
(counting multiplicities) doesn’t mean they’re easy to find! The roots of a first degree 
polynomial p(z) = ao + aız are trivial, z = —ao /aı. The roots of a quadratic are given 
by the quadratic formula, which involves a finite number of basic operations (add, sub- 
tract, multiply, divide) and square roots. The roots of a general cubic are solutions to 
aZ? + aoz* + a,z + ao = O. There is a formula for these roots, discovered in the 16th 
century. The solution to the cubic involves a finite number of the basic four arith- 
metic operations applied to the coefficients ao, a1, a2, a3, as well as a finite number 
of square and cube roots. The formula for the roots of a general quartic equation, 
a4z + a3z? + azz? + aıZ + ao = O is similar, requiring only a finite number of basic 
arithmetic operations, as well as square and cube roots. It was also discovered in the 
16th century. 

Any hope for a similar formula for the roots of higher degree polynomials was 
dashed in the 18th century by the work of Evariste Galois, Abel, Ruffini, and others. 
They showed that the roots of a general 5th or higher polynomial cannot be expressed 
using a finite number of additions, subtractions, multiplication, divisions, powers, and 
nth roots of the polynomial’s coefficients. Nonetheless, we can always approximate 
roots numerically, for example, with Newton’s method. 


Application: Partial Fractions 


The computations of partial fraction decompositions from Calculus 2 (or MA 211) are 
conceptually a little simpler if we work with complex numbers, though we don’t gain 
much in computational efficiency. The conceptual simplicity comes from the fact that 
any polynomial factors completely into linear pieces in the complex numbers, as in 
(A.53). This isn’t the case if we work in the real numbers. 
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Example 150 Factor p(x) = x? — x? + x — 1 over the real numbers and the 


complex numbers. 


SOLUTION Over the real numbers p factors as p(x) = (x —1)(x* + 1). But the 
quadratic piece does not factor, since it has no real roots. However, if we think of p as 
a function of a complex variable, p(z) = z? — zZ? +z — 1, then p factors completely as 


p(z) = (z-1)(z-f)(z +i). (A.57) 


As mentioned, this can make partial fraction decompositions a little easier from a 
conceptual, if not computational point of view. Let’s do an example to start. 


A.4 More on Functions of a Complex Variable and Calculus 


Example 151 Find a partial fraction expansion for the function f(x) = poe. 
SOLUTION If we restrict x to be a real number we factor the denominator as 


(x — 1)(x? + 1) as in Example 150. The usual rules for partial fractions from Calculus 
2 indicate we should try 


2x? + 4x —2 A Bx +C 
+ 


~e—x2+x-1 x-1 x241° 


We next find a common denominator, match powers of x in the numerator, and solve 
for A,B,C to find A = 2, B = 0, and C = 4. Thus 


2x +4x-2 2 4 


; A.58 
x—x2+x—-1 x I eat ( ) 


After obtaining the decomposition (A.58) we’d be in a position to, for example, 
integrate f(x), by integrating each piece in the decomposition, or perform an Inverse 
Laplace Transform, or whatever computation we’re interested in. 

Now let’s redo Example 151 but working over the complex numbers; we’ll switch to 
Z for our variable. We don’t have to remember special rules for linear versus quadratic 
factors in the denominator. 


Example 152 Compute the partial fraction expansion for f in Example 151 by 
factoring the denominator over the complex numbers. 


SOLUTION With f as in Example 151 we can factor z? — z? +z — 1 = (z — 
1)(z + /)(z — i) and so try an expansion of the form 


27 + 4z—2 A2 B2 Co 
f=- = a ee re 


B—-wv+7—-1 z—1 z-i z+i 


Getting a common denominator on the right yields 


2z? +4z-2 Ad B2 be (0 
z-z +z—-1 z-1 z-i z+i 
(A2 + B2 + C2)z? + (1 — i)(B2 + iCz)z + (A2 + i(C = B>)) 


z—z?+z-1 


To make this match f(z) we need to match the denominator 2z? + 4z — 2 of f(z), which 
leads to 


A2 + B2 +02 =2, (1—i)(B2+iC2)=4, A+ i(C2 — B2) = —2. 


This is three linear equations in three complex unknowns. The solution is A, = 2, By = 
2i, Cp = —2i. That is, 


2z? +4z—-2 2 , 2 2i (A.59) 
z-z +z-1 z—-1 z+i z-i i 
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Compare this to (A.58). In (A.59) all denominators are linear in z. 


More generally, suppose we have a rational function (a function of the form f(z) = 
p(z)/q(z) where p and q are polynomials) in which the degree of p is strictly less than 
the degree of q. Suppose q has r distinct complex roots 4,--- , 4 where @ has multi- 
plicity mę, and none of these are roots for p. Then f(z) has a partial fraction expansion 


plz). & A m B Mm R 
k k k 
fz) == = i ++ — (A.60) 
q(z) 2 C- Ay > z-AF 2 z- AF 
for some (complex) coefficients A1, . . . , Am, , B1, - < - , Bm; - <- Ri; -< <, Rm,- 
Example 153 Compute the partial fraction expansion for f(z) = aa 
SOLUTION The denominator factors as 


z +22 +1= (2241) = (z-i)? (z+ i}?. 
Based on (A.60) we try an expansion of the form 


4z Ai A2 + Bı + B2 
z +224+1 z—i (z—-ił z+i (z+) 


Getting a common denominator on the right and matching powers of z?, z+, z*, z? leads 
to four equations in four unknowns, A1, A2, B1, B2. (algebra omitted). The solution is 
Ai 0, A2 —i, By 0, By i, So 


4z O i 4 i 
O+274+4 8 (z—f2 (z+) 


Although the computations involved in a partial fraction expansion are (still) cum- 
bersome, the concept embodied by (A.60) is important. In particular, the roots of the 
denominator q(z) ina rational function f(z) = p(z) /q(z) a called the poles of the func- 
tion f(z). They are a bit like vertical asymptotes for functions of a real variable. In 
many physical and engineering applications what we really care about is the number 
and location of the poles of f in (A.60), quite apart from the values of the coefficients. 
The poles contain a lot of information about the function. 


Calculus and Functions of a Complex Variable 


In Calculus 1 you learned the definition for the derivative of a real-valued function f(x) 
of a real variable x, as well as the various rules for computing derivatives. But if f is 
a function that accepts a complex argument (and puts out a complex answer) do any 
of these things still make sense? The short answer is “yes.” The derivative still makes 
sense via the standard definition, and the usual rules (product, quotient, power, chain) 
still hold. If you’re content to accept this, you can stop reading. 


A.4 More on Functions of a Complex Variable and Calculus 


But differentiating functions of a complex variable leads to some interesting twists 
that don’t occur in the real case. Recall the definition of the derivative from Calculus 
1: we say that a function f(x) is differentiable at x = xo if the limit 


him f(xo + Ax) — f(xo) 


Ax—0 Ax ed) 


exists. We then call this limit f (xo). The statement Ax — Ocan be quantified carefully, 
but the important point to note is that Ax can assume positive values, negative values, 
or both, as it approaches 0; it’s not enough for the limit to exists if Ax approaches zero 
in some special way, e.g., through only positive values. This puts some restrictions on 
what functions are differentiable. For example, f(x) = |x| is not differentiable at x = 0, 
forif A — O+ (through positive values) the limit in (A.61) is 1, but if Ax — O7 (through 

negative values) the limit in (A.61) is —1. 
For a function f(z) of a complex variable z we will say that fis differentiable at z = Zo 

if the limit 
ie f(Zo + Az) — f(zo) 


A.62 
Az—0 Az ( ) 


exists. If so, we call this limit f (zo). Here Az = Ax + i^y is a complex number. The 
statement that Az — 0 can be taken to mean that the modulus |Az| — 0 (as a real 
number). 

The definition of differentiability for a function of a complex variable is more re- 
strictive than its real counterpart. There are many ways that Az can approach O, e.g., 
along the real axis, along the imaginary axis, along a real multiple of 1 + i, a spiral, 
etc. This makes it more difficult for a function of a complex variable to be differen- 
tiable. But functions that actually turn out to be differentiable have many wonderful 
properties. 

All the elementary functions you know and love—rational functions, trig functions, 
exponential, logs—are in fact differentiable when extended to the complex plane. We 
won't prove this here, but in many cases the proofs from elementary calculus work 
without modification. Let’s look at a simple representative example. 


Example 154 Let f(z) = z?. Compute f (z) by using (A.62). 


SOLUTION We have 


f(zo + Az) —f (zo) (zo + Az)? — z2 
Az—0 Az Az—0 Az 
_ 2zgAz + (Az)? 
= lim — 
Az—+0 Az 
2Zo + Az) 


lim ( 
Az—0 
= 2Zo. 


Sof’ (Zo) = 2Zo. This proof is exactly as in the real case. The same is true for any power 
of z, as well as linear combinations of powers. Thus polynomials of a complex variable 
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are differentiable, and the same differentiation rules from Calculus | still apply. 


Similar arguments can be made to show that familiar rules for exponentials, logs, 
function inverses, products, quotients, and the chain rule still hold. Techniques such 
as power series and integration extend as well, and become even more powerful than 
inthe real case. If you want know more and see the power and utility of doing calculus 
with functions of a complex argument (a subject called Complex Analysis) you should 


take MA 367. 


Exercises A.4 


1. Find all the roots of zZ? + 2z4+1=0. 
(Note this is quadratic in z*.) You should 
get 8 solutions, some of which may be 
repeated. What is the multiplicity of 


each root? 


2. Find the roots of each polynomial equa- 
tion below, as well as the multiplicity of 


each root. 


(a) “+1=0. 


(b) (f +3)? +2(°+3)4+1=0. 


10 


(c) r 
roots in polar form. 
(d) Pf +r =0. 


(e) °° — 2° +1 = OHint: express 
the roots in polar form. 


3. Assume that the rules for differentiat- 
ing exponentials and also the chain rule 
hold. Use (A.50) and (A.51) to verify 
that the usual rules for the derivative of 


sine and cosine still hold. 


4. Itis a fact that if all the roots of a poly- 
nomial p(z) lie in the right half plane 
Re(z) > O then so do the roots for 
p’(z). (This fact holds in a much more 


general setting). 


Illustrate this fact by considering the 
polynomial p(z) = (z — 2)(z — (2+ 
i) )(2— (2-1) (2—(3+2/))(2— (3-21) 
(a 5th degree polynomial with roots z = 
2,2 +i,2 —i,3 + 2i,3 — 2i, all in the 
right half plane). Compute the roots of 
p’(z) and verify all are in the right half 
plane (have positive real part). 
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+7°° = 0. Hint: express the 


5. Recall from Calculus 2 that for the func- 


tion 1/x of a real variable x we have an- 
tiderivative 


[Samui +e 
x 


The absolute value is necessary for the 
formula to be valid for x < 0, since 
In(x) makes no sense if you only work 
in the real numbers. But you may 
have noticed that computer algebra 
systems (Maple, Mathematica) give the 
antiderivative as just In(x), with no ab- 
solute value. Why? Follow the steps 
below to find out. 


As mentioned, the absolute value is 
needed if the antiderivative In |x| is to 
be used for values of x < 0. But now 
that we know how to compute In(x) for 
x < 0, let’s see what happens if we drop 
the absolute values and just try In(x) as 
an antiderivative. 


(a) Use (A.48) to show that if x is real 
and x < 0 then 


In(x) = In(—x) + im. (A.63) 


Note that In(—x) here is the usual 
logarithm of a positive real. 


(b) Differentiate the right side of 
(A.63) with respect to x and show 
that the result is indeed 1/x. So 
In(x) without absolute values is a 
valid antiderivative for 1/x, even 
ifx <0. 


SOLUTIONS TO SELECTED PROBLEMS 


Chapter 1 17. Ri +R) > Rp 
bs 19. Ry +> Ro 
Section 1.1 
21. x=2,y=1 
ly 23. x= —1,y=0 
3. y 25. xı = —2,x2 = 1,x3 = 2 
5. n i 
Section 1.3 
7. y 
9. y 1. (a) yes (c) no 
11. x= 1,y = —2 (b) no (d) yes 
13. x= —1,y = 0,z = 2 
15. 29 chickens and 33 pigs 3. (a) no (c) yes 
Section 1.2 (b) yes (d) yes 
3 4 5 7 E [1 0 
1 —1 1 —3 1 , | 0 1 
2 —2 3 5 m 
7 1 3 
1 3 -4 5 17 ' | 0 0 
3. /-1 0 4 8 1 j 
2 3 4 5 6 9 |i 0 3 
JO 1 7 
5 x +2x%x = 3 3 
i —xX1, + 3x2 = 9 11 1 —1 2 
: 0 0 
7 X1 +X —X3—-X4 = 2 n 
© 2x1 + x2 + 3x3 + 5xg = 7 1 0 0 
g X +x +7% = 2 13. x g ; 
© x +3x3+2x, = 5 
2 -1 7 1 0 0 
11. |5 0 3 15. jO 1 0 
0 4 -2 0 0 1 
2 —1 7 1 0 O 1 
13. 0 4 —2 17. o 1 1 1 
5 8 —1 o 0 0 0 
2 —1 7 1 0 1 3 
15. 0 4 —2 19. 0 1 -2 4 
0 5/2 —29/2 00 0 o0 
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1 1 0 0 0 0 
21; 0 O 1 3 1 4 
Section 1.4 
1. x, = 1 — 2x2; X is free. Possible solutions: 
xı = 1, x2 = O and x, 1,X%x = 1. 
3. xX, = 1x. =2 
5. No solution; the system is inconsistent. 
7. xı = —11 + 10x3; x2 = —4 + 4x3; x3 is 
free. Possible solutions: x; = —11, 
X2 4, x3 = O and x, 1, x2 = 0 and 
x3 = 1. 
9. xı = 1 — x2 — X4; X2 is free; x3 = 1 — 2x4; 
X4 is free. Possible solutions: x; = 1, 
X2 = 0,x3 = 1, x4 = O and x1 2, 
X2 = 1 = = 2 

11. No solution; the system is inconsistent. 

13. x1 = i — 4x3; x2 = A — 3X3; X3 is free. 
Possible solutions: x1 = ix = E, x3 = 0 
and x, = —1,x2 = 0, X3 = 1 

15. Never exactly 1 solution; infinite solutions if 
k = 2; no solution if k # 2. 

17. Exactly 1 solution if k Æ 2; no solution if 
k = 2; never infinite solutions. 

Section 1.5 
1. 29 chickens and 33 pigs 
3. 42 grande tables, 22 venti tables 
5. 30 men, 15 women, 20 kids 
7. f(x) = —2x + 10 
9. f(x) = {x +3x+1 

11. f(x) =3 

13. f(x) =x? +1 

15. f(x) = 3x+1 

17. The augmented matrix from this system is 


1 1 1 1 8 


6 1 2 3 24 |. From this we 


Oo 1 —1 0 0 
find the solution 

8 1 

toe 
3 3f 
8 1 

) 
3 3f 
8 1 

w= — — -f. 
3 3 


The only time each of these variables are 
nonnegative integers is when f = 2 or 


19. 


21. 


23. 


f = 8. Iff = 2, then we have 2 
touchdowns, 2 extra points and 2 two point 
conversions (and 2 field goals); this doesn’t 
make sense since the extra points and two 
point conversions follow touchdowns. If 

f = 8, then we have no touchdowns, extra 
points or two point conversions (just 8 field 
goals). This is the only solution; all points 
were scored from field goals. 


Let x1, x2 and x3 represent the number of 
free throws, 2 point and 3 point shots 
taken. The augmented matrix from this 


system is eae ae al From this we 
y 1 2 3 3V 
find the solution 

X1 = —20 + x3 

X2 = 50 — 2x3. 


In order for xı and x2 to be nonnegative, we 
need 20 < x3 < 25. Thus there are 6 
different scenerios: the “first” is where 20 
three point shots are taken, no free throws, 
and 10 two point shots; the “last” is where 
25 three point shots are taken, 5 free 
throws, and no two point shots. 


Let y = ax + b; all linear functions through 
(1,3) come in the form y = (3 — b)x + b. 
Examples: b = O yields y = 3x;b = 2 
yields y = x + 2. 


Let y = ax? + bx + c; we find that 
a= -4 + fcandb = ; — 3c. Examples: 


c = 1yields y = —x + 1; c = 3 yields 
y =x — 4x43. 


Chapter 2 


Section 2.1 
a ees 
a Nias e 
ekea 
7. es 
=. [Ts 
H E E 
Ee 
15. a=2,b=1 


17. a = 5/2 + 3/2b 
19. No solution. 
21. No solution. 


Section 2.2 


1. —22 
0 


—2 


11. Not possible. 


13. AB = 5 A 


|10 -9 
-3 24 
BA = |a A 
f-1 -2 12 
15. AB=| o 4 A 


BA is not possible. 


17. AB is not possible. 


27 —33 39 
ac ey e3 a 
—32 34 —24 
19. AB = | —32 38 —8 
—16 21 4 
22 —14 
AS E =i 
—56 2 —36 
21. AB= 20 19 —30 
—50 -13 0 
—46 40 
DAT | 72 9 | 
15 22 21 1 
oe | 16 —53 —59 i 
BA is not possible. 
0 0 4 
25. AB=|6 4 -2 
2 -4 -6 
2 —2 6 
BA= |2 2 4 
4 0 —6 
21 -17 —-5 
27. AB= | 19 5 19 
5 9 4 
19 5 23 
BA = 5 —7 -1 
—14 6 18 


4 -6 
29. AD= $ = 
4 8 
HER 
2 2 2 
31. DA = 6 6 6 
—15 —15 —15 
2 —3 5 
AD = 4 —6 10 
—6 9 —15 
dia dıb dic 
33. DA= dod de dof 
d3g d3h d3i 
dia dəb d3c 
AD = | dd dze asf 
dig dəh d3i 
a —6 
35. AX = | A 
—5 
37. AX= 5 
21 
X1 + 2x2 + 3x3 
39. AX = X1 + 2x3 
2x1 + 3x2 + x3 
4 0 8 0 
Ds AAs 
Oh Dor p 54 = k A 
o oO 1 1 0 0 
43. A2=]1 0 O}];A27=1]0 1 0 
1 0O 0 oO 1 
fo -2 
45. (a) e Er 
10 2 
(b) | 5 al 
f—11 -15 
ae ae | 
(d) No 
(e) (A+B)(A+B) = AA+AB+BA+BB = 
A? + AB + BA + B?. 
Section 2.3 


1. 


Ea -1/5 => 3 
z+7=| {| z-7=|3,| 


Sketches will vary depending on choice of 
origin of each vector. 
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x+y and X are the same length (they just point 
in opposite directions). 
E 13. (a) IXI] = v2; |Iy|| = V13; 
y X+y\| = 
x-y (b) ||x|| = v5; ||7]| = 3v5; 
fa X +y] = 4v5 
(9 [1#] = VIG; ll] = v35; 
X+ yl| = v65. 
E We ees 1 (d) ||x|| = v5; |||] = 2v5; 
3. X+y= 3 | X-Y=]_4 X+y\| = V5. 
The equality holds sometimes; only when X 


Sketches will vary depending on choice of 


origin of each vector. and y point along the same line, in the same 


direction. 


= 


1 Ay 


xı 
>< 


X+V 
y 
5. Sketches will vary depending on choice of À 
origin of each vector. y AX 
X+y A 
x 
15; 
=, y 
Y 4 
X 
x 
=y 
AX 
=y Y y 
17. ay. 
Section 2.4 


7. Sketches will vary depending on choice of 
origin of each vector. 


x+y 


. Multiply AŬ and AV to verify. 
. Multiply Ad and AV to verify. 


. Multiply Ad, AV and A(i + V) to verify. 


1 

3 

5. Multiply Ad and AV to verify. 

7 

9. Multiply AŬ, AV and A(u + V) to verify. 


— fo 
11. (a) xX = H 
~_ | 2/5 
tO) as | 
> x -_ To 
9. ||x|| = v5; |ax|| = V45 = 3V5. The 13. (a) X= a 
vector ax is 3 times as long as X. + 
“ 2 
11. ||x|| = v34; ||ax|| = V/34. The vectors ax (b) X= E 
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15. 


17. 


19. 


21. 


5/4 
(a) X=x3] 1 
1 


1 5/4 
(bb) ¥=|0] +x 3] 1 
0 1 


2 —1 
(a) x = x3 a? + x4 ae 
0 1 
(b) X= 
—2 2 —1 
gh sigs os rer ah 
0 (0) 1 
—1/2 1/2 
1 0 
(a) KX = x2 0 + xq | —1/2 | + 
0 1 
0 0 
13/2 
0 
X5 —2 
0 
1 
—5 —1/2 
0 1 
(b) X= | 3/2 | +x2 0 + 
0 0 
0 0 
1/2 13/2 
0 0 
X4 —1/2 + x5 —2 
1 0 
0 1 
1 0 
13/9 —1 
(a) X = xq 1/3 | +xs 1 
1 0 
0 1 
(b) x= 


< 


y 
v 
<4 EA > X 
Xp 
Section 2.5 
[1 -9 
1 x=| 4 =| 
Rade -57 
3. X=] 5 z] 
[-5 2 -3 
a [ae E El 
f1 0 
TAE S A 


| 5/3 2/3 
11. X= | —1/3 1/6 


1/3 1/3 
Section 2.6 
[ -24 —5 
t | 5 1 | 
[1⁄3 o0 
3 
[0 1/7 
5. A~! does not exist. 
[1 0 
4 | 0 ‘| 
[—5/13 3/13 
9 
| 1/13 2/13 
[ —2 1 
Fe | gps Ea 


1 2 2 
13. }0 1 —3 
6 10 —5 
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1 0 0 11. Likely some entries that should be 0 will not 
15. R i ; be exactly 0, but rather very small values. 
Section 2.8 
17. ATİ does not exist. 
25 8 0 X 
19. | 78 25 0 3. 
—30 —9 1 
5; 
0O 1 0 7 
21. |O 0 1 : 
1 0 0 9. 
11. 
1 0 0 0 
= = = 13. 
ae 3 1 0 4 
—35 -10 1 —47 15. 
—2 -2 0 -9 
17. 
28 18 3 —19 19. 
5 1 0 -5 
235 Ja s5 a o0 Chapter 3 
52 60 12 —15 
Oe or eG Section 3.1 
0 0 0 1 
ah 1 0 0 0 1. Ais symmetric. | 7 zl 
0O 1 0 0 
2 3. Ais diagonal, asis A'.[ 9 J 
29. = 0 9 
3 
i E 3 z| 
ae -9 1 -8 
1 
4 -9 
Z7 7 —7 6 
33. X= 1 i —4 3 
—1 =f =9 
3 —7 
35. X= | -1 gas 
—9 2 
-3 
Section 2.7 
-9 6 a 
11. 4 -3 1 
—2 3 
a 
1. (AB) -| ; a 10 -7 -1 
29/5 18/5 T a 
-1 — =, 13. Aissymmetric. | 0 2 3 
3: AB) E: 7/5 | y eae 
5 DE z) Ji ee CF 
-1 -3 15. | -5 5 —4 
—3 5 —3 —6 —10 
| 1 fl 
A 4 5 -6 
17. | 2 -4 6 
-3 7 
=o 
7. A -| 1 eA -9 -10 9 
AcN)-1 2 7 19. Ais upper triangular; A’ is lower triangular. 
Ce eo -3 0 0 
—4 -3 0 
9. Solutions will vary. S60 ge ES 
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21. Aisdiagonal,asisA’.] 0 2 O 


0o 0 -1 
—1 2 
23. A is skew symmetric. 1 0 —4 
—2 4 0 
Section 3.2 
1. 6 
3. 3 
5. —9 
Zed 
9. Not defined; the matrix must be square. 
11. —23 
13. 4 
15. 0 
17. (a) tr(A)=8; tr(B)=—2; tr(A + B)=6 
(b) tr(AB) = 53 = tr(BA) 
19. (a) tr(A)=—1; tr(B)=6; tr(A + B)=5 
(b) tr(AB) = 201 = tr(BA) 
Section 3.3 
1. 34 
3. —44 
5. —44 
7. 28 
9. (a) The submatrices are ue 5 
6 10 
3 6 3 7 
i fo ara k Al 
respectively. 
(b) C12 = 34, C12 = —24,Ci3 = 11 
11. (a) The submatrices are È cal 
E a and | -6 d 
-9 9 -9 3)’ 
respectively. 
(b) C12 3, Cir = —63, C1,3 = 18 
13. —59 
15. 15 
17. 3 
19. 0 
21. 0 
23. —113 


25. Hint: C11 = d. 
Section 3.4 
1. 84 
3. 0 
5. 10 
7. 24 
9. 175 
11. —200 
13. 34 
15. (a) det(A) = 41; R2 + R3 
(b) det(B) = 164; —4R3 — R3 
(c) det(C) = —41; Rp + Ri > Rı 
17. (a) det(A) = —16; Ri + Ro then 
Ri + R3 
(b) det(B) = —16; —Ri — Ri and 
—R > Rp 
(c) det(C) = —432; C = 3 * M 
19. det(A) = 4, det(B) = 4, det(AB) = 16 


21. 


det(A) = —12, det(B) = 29, 
det(AB) = —348 


23. —59 
25. 15 
27. 3 
29. 0 
Section 3.5 
1. (a) det(A) = 14, det (A1) = 70, 
det (A2) = 14 
> 5 
ar-f] 
3. (a) det(A) = 0, det (Ai) = 0, 
det A2) =0 
(b) Infinite solutions exist. 
5. (a) det(A) = 16, det (A1) = —64, 
det (A2) = 80 
= —4 
"r 
7. (a) det(A) = —123, det (A1) = —492, 
det (A2) = 123, det (A3) = 492 
4 
(b) X= | —1 
—4 
9. (a) det(A) = 56, det (A1) = 224, 


det (A2) = 0, det (A3) = —112 
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4 
(b) x= | O 
—2 
11. (a) det (A) = 0, det (A1) = 147, 
det (A2) = —49, det (A3) = —49 
(b) No solution exists. 
Chapter 4 
Section 4.1 
1. A= 3 
3. =0 
BASS 
st 
ela 
3 
9. X= | -7 
7 
—1 
11. X= 1 
1 
e 9 
13. A1 = 4 with ṣi = hel 
A2 = 5withx> = H 
oho —2 
15. Ai = —3 with xi = | | 
A2 = 5withx> = [$] 
ae at 1 
17. A1 = 2 with ṣi = EE 
wai os —1 
da = 4 with = | | 
Seco 1 
19. A, = —1 with xi = Ee 
à2 = —3 withx, = l 
2= 2=] o 
—3 
21. A. =3withxļi = | 0 |; 
2 
—5 
A2 = 4withx, = | —1 
1 
1 
às = 5withx3 = | 0 
0 


24 
23. ` = —Swithx; = | 13 |; 
8 
6 
àz = —2 with% = | 5 
1 
0 
A3 = 3withx3 = | 1 
0 
0 
25. ` = —2withx; = | 0]; 
1 
0 
à: = 1withxs = | 3 
5 
28 
A3 = 5withx3 = | 7 
1 
1 
27. ` =—2withx= | 0 l; 
1 
1 
à: = 3with¥ = | 1]; 
1 
0 
A3 = 5withXx = | 1 
1 
Section 4.2 
itt 2 [sd 
1. (a) à = 1withx; = ft 
ESEN 
A2 = 4withx = | 
Pee a 
(b) Ar = 1 with xi = 1 |: 
A2 = 4with x3 = ie. 
2= 2= | 4 
TEE. 1 
(c) Ar = 1/4 with xy = pi 
à2 = 1withx = d 
(a) 5 l 
(e) 4 
3. 


(a) who = ae 
2 = 0 with xo = F 
(b) dy = —2 with 3 = | 
1 

5 


à2 = 0 with x = | 


(c) Ais not invertible. 


(d) -1 
(e) 0 


—7 
5. (a) A. = —4 with xi = È 3 
0 
à: = 3withxs = | 0 
1 
9 
uk 


A3 = 4 with x3 = 


22 
—1 
(b) Ay = —4with ṣi = | 9 |; 
0 
—20 
à2 = 3withxa = | 26 
7 
—1 
A3 = 4 with x3 = 1 
0 
—7 
(c) Ar = —1/4 with ṣi = | —7 |; 
6 
0 
A2 = 1/3 with x2 = | 0 
1 
9 
A3 = 1/4withx3 = | 1 
22 
(d) 3 
(e) —48 
Section 4.3 


1. Double eigenvalue \ = 2, eigenvectors 
1 
0 


and | n of multiples thereof, or anything 

in the plane. 
3. Double eigenvalue \ = —1, eigenvector 

[2] 
2 

of multiples thereof. 

5. Eigenvalues \ = —3 + 2i, eigenvectors 
| —1+ i 
2 -2i 


, A = —3 — 2i, eigenvectors 


—1- 2i 
2+2i 


7. Eigenvalue à = 2, algebraic multiplicity 3. 
1 0 
Only two eigenvectors, | O | and | 0 
0 1 
9. Eigenvalue \ = 4, algebraic multiplicity 3. 
—1 
Only one eigenvector, 3 
2 
0 
11. Eigenvalue à = 3, eigenvector | O |, 
1 
eigenvalue \ = —3 + 6i, eigenvector 
—1 — 8i 
—4 — 2i |, eigenvalue \ = —3 — 6i, 
5 
—1+ 8i 
eigenvector | —4 + 2i 
5 
Chapter 5 
Section 5.1 
aagi 
IAT [3 4] 
RT. 3 
ASi a 
Feya] 
5. A= |2 1] 
ian S84 
TAS | 
fo -1 
9. A= Bee =| 
11. Yes, these are the same; the transformation 
matrix in each is =E 30 
0 —1 


13. Yes, these are the same. Each produces the 
; _ [1/2 0 
transformation matrix . 
0 3 
Section 5.2 
1. Yes 
3. No; cannot add a constant. 
5. Yes. 
1 2 
7. ([T]=]3 -5 
0 2 
1 0 3 
9. [T]=|1 0 -1 
1 0 1 
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Section 5.3 
3 —1 
1. X+y= |2|,š—-yïy= |—4 
4 (0) 


Sketches will vary slightly depending on 
orientation. 


X— 
4 —2 
3. X+y=]4)],%-Y= | -2 
8 —4 


Sketches will vary slightly depending on 


orientation. 


5. Sketches may vary slightly. 
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. The roots are x = — > 4 


2X 
z- 
X+y 
Z 
-Y 
y 
x 
7. Sketches may vary slightly. 
Z 
=> y 
Z+ x: 
x 
¥-7 


9. ||x|| = v30, ||ax|| = 120 = 2/30 
. [IXI] = V54 = 3v6, 
||ax’ 


ax|| = V270 = 15/6 


Section A.1 


. (a): 2,3, neither; (b) e, 7, neither; (c) 0, 5, 


pure imaginary; (d) —7, —5, neither. 


. Starting with i2 we have 


—1, —i, 1, i,—1, —i, 1. Since 219 is equal to 
3 modulo 4, i249 = P = —i. 


. (a) 


Zi +z: =% + Z = overlinez; + z2 — 4; 
(b) Z4 — 322 = 29 — 3i, 
Azs — 3z1 + 2z3 = 24 + 9i; (c) —1 — 6i. 


. (a): 27/65 — 311/65; (b) —6/13 — 9i/13; 


(c) —26/3; (d) —1/975 — i/1300. 
ba Vb? —Aac | if 


2a ~~ 2a 
b? — 4ac both roots are real. If 


b? — 4ac < 0 roots can be written as 


.\/4ac—b2 : 
x = — 2 + iX conjugate. 


“Sage 2a 


11. Start with 


cp — dq = a and cq + dp = b. 


Multiply the first equation by c, the second 
by d, and add to find (c? + d*)p = ac + bd, 
so p = (ac + bd) /(c? + d*). Similarly 
multiply the first equation by d, the second 
by c, and subtract to find 

—(c? + d?)q = ad — bc, so 

q = (bc — ad) /(c* +d”). 


13. The roots are approximately 


1.122 + 0.308/, —0.673 + 1.421, 0.898. 


Two conjugate pairs, one real root. 


Section A.2 


1. (a) |z| = |Z| = V2, arg(z) = 7/4, arg(z) = 


—1/4; (b) 

|z| = |z| = 7, arg(z) = 0, arg(z) = 0; (c) 
|z| = |z| = 3V2, arg(z) = 37/4, arg(z) = 
—31/4; (d) 

|z| = |z| = 5, arg(z) = 7, arg(z) = 7; (e) 
|z| = |z| = 1, arg(z) = 7/2, arg(z) = 

— 1/2; (f) |z| = |Z| = 2, arg(z) = 

a /3, arg(Z) = —7:/3. In each case 

arg(z) = —arg(Z), except for the razor’s 
edge case when z is real and negative. 


. It’s just vector addition and subraction. 

. (a)r = 3,0 = 0; (b) r= V2, 0 = 17/4; (c) 
r = 2V2, 0 = 30/4; (d) r = 1,0 = 1/2; 
(e) r= 7,0 = 7; (f) 
r=5,0=37-27=7. 


. Ifz = x + iy then Z = x — iy. Multiplying 
produces zZ = x? + xyi — xyi + y? = |z|?. 


. Write z in polar form, as z = e?™!/3, Then 
z? = e?"i = 1. For the second case write 

z = e4"/3 Then Ż = e^"! = 1. And 
trivially, we can take 

z = cos(07/3) + isin(07/3) = 1 and note 
that 13 = 1. 


Section A.3 


1. (a) 1,/,—1, ~i; (b) 


1+ i, (V3 + 1)/2 + i(v3 
1)/2, (—V3 + 1)/2 — (v3 + 1)/2; (c) 
v2 ( 1 i) (all combinations); (d) 
2, 2i, —2, —2i. 


3. It’s clear that if z = e2*/" then 


z” = ek" — 1, so z satisfies z” — 1 =0. 
But if k is not a multiple of n then z 4 1, so 
if 

0 = 1-2" = (1—z)(14+z4+2?+---+2"-1) 


we must conclude that 
14z4t24---271=0. 


5. (a) In(3); (b) i (c) 1 — i (d) in/2; (e) 


In(7) + im; (f) In(5) + ir. 


7. We have |iy| = |illy| = |y| and 


arg(iy) = arg(i) + arg(y) = it /2, so 
In(iy) = |y| + iz /2. 


Section A.4 


1. Letu = z4, so the polynomial becomes 


u + 2u? +1 = Owith double root u = —1. 


We then have to solve z* = —1, with roots 
2 
Sen 


(all 4 sign combinations). Thus 


z + 2z4 + 1 = 0 has 4 distinct roots, each 


of multiplicity 2. 


3. We have cos(z) = e7 /2 + e7? /2. The 
derivative of the right side is 
ie” /2 — ie™?/2 = — (e? — e7?) /(2i) = 
— sin(z). Similarly 


sin(z) = e” /2i + e7" /2i. The derivative of 


the right side is e” /2 + e—?/2 = cos(z). 


5. (a) We have In(x) = In |x| + iarg(x). If 
x < Othen |x| = —x and arg(x) = iz, so 
In(x) = In(—x) + iz. (b) The derivative of 
In(—x) with respect to x is 4 -(—1) = 2 
by the chain rule. The derivative of iz is 
zero, so the derivative of In(x) is 1/x. 
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i, 248 
j, 248 


addition, complex, 249 
antisymmetric, 142 
argument, 259 

principal value, 259 
augmented matrix, 8 
axis 

imaginary, 258 

real, 258 


back substitution, 6, 9 
basic variable, 29 


characteristic polynomial, 181 
cofactor, 151 

expansion, 153, 160 
complex addition, 249 
complex conjugation, 252 
complex division, 251 
complex exponentiation, 256 
complex multiplication, 250 
complex number, 248 
complex part, 248 
complex powers, 250 
complex roots, 269 
complex square roots, 251 
complex subtraction, 249 
conjugation, 252 
conjugation, polar form, 268 
consistent, 27, 92, 94 
cosine of complex number, 275 
Cramer’s Rule, 173 


determinant 
3 x 3 shortcut, 169 


and elementary row operations, 163 


definition, 155 
of 2 x 2 matrices, 150 
of triangular matrices, 162 
properties, 168 
diagonal, 122 
definition, 137 
differentiable function, 281 
division, complex, 251 
division, polar form, 267 


eigenvalue 
definition, 178 
finding, 182 
properties, 196 
real matrix, 205 
eigenvector, see eigenvalue 
elementary row operations, 15 
and determinants, 163 
Euler’s identity, 255 
exponentiation, 256 
complex, 256 


free variable, 28, 29 
Fundamental Theorem of Algebra, 277 


Gauss-Jordan elimination, 12, 17, 20 
Gaussian elimination, 12, 17, 20 
backward steps, 19 
forward steps, 19 
Gaussian elimination with back 
substitution, 12 


homogeneous, 91, 92, 94 


identity matrix, 65 
imaginary axis, 258 
inconsistent, 27 
inverse 
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Index 


computing, 111 partial fraction decomposition, 278 
definition, 110 particular solution, 32 

Invertible Matrix Theorem, 117 pivot, 16 

properties, 117, 122 polar form, 260 

uniqueness, 110 polynomial, 276 


Invertible Matrix Theorem, 117, 168,197 powers, complex, 250 
principal value of argument, 259 


leading one, 16, 30, 31 problem solving, 40 
linear combination, 125 pseudoinverse, 144 
linear dependence, 127 pure imaginary, 248 


linear equation, 3 


linear transformation rank, 36 
and 0, 228 real, 248 
conditions on, 233 real axis, 258 
definition, 224 real part, 248 
standard matrix of, 227, 229 rectangular form, 248 
logarithms, 273 reduced row echelon form, 15, 16 
principal branch, 273 root 
multiplicity, 277 
matrix roots, 269 
addition, 51 polynomial, 276 


arithmetic properties, 53 row echelon form, 11, 15, 16 


augmented, 8 
cofactor, 151 
definition, 7 
determinant, 150, 155 
diagonal, 137 


sine of complex number, 275 

skew symmetric, 142 
definition, 142 
theorem, 143 


: solution, 3 

equality, 50 infinite, 26, 30, 94 

identity matrix, 65 none, 26 

inverse, 110, 111 particular, 32 

minor, 151 types, 26 

multiplication, 58 unique, 26, 94, 113 

properties, 66 standard unit vector, 228 

rank, 36 subtraction, complex, 249 

scalar multiplication, 51 superposition, 125 

square, 7 symmetric, 142 

the zero matrix, 53 definition, 142 

transpose, 136 theorem, 143 

triangular, 137 system of linear equations 
minor, 151 consistent, 27, 30, 92, 94 
modulus, 259 definition, 3 
multiplication, complex, 250 homogeneous, 91 
multiplication, polar form, 267 inconsistent, 27, 31 
multiplicity, 277 solution, 3 
Parallelogram Law, 73 trace 
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definition, 145 
properties, 147 
transpose, 135 
definition, 136 
properties, 140 
skew-symmetric, 142 
symmetric, 142 
triangular matrix 
definition, 137 
determinant, 162 


variable 
basic, 29 
dependent, 29 
free, 28, 29 
independent, 29 
vector 
column, 56 
length, 78, 244 
row, 56 


zero matrix, 53 
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